aboutsummaryrefslogtreecommitdiff
path: root/weed
AgeCommit message (Collapse)AuthorFilesLines
2024-07-26Revert "Check `ShouldGrowVolumes` before returning error in assign. (#5819)"chrislu2-2/+2
This reverts commit 98d66338d0333cd955f7840c64ef95e3c4807a17.
2024-07-26fix delete chunk failed if volumeSever specified grpc.port (#5820)wyang3-6/+17
Co-authored-by: Yang Wang <yangwang@weride.ai>
2024-07-26same change as #5819chrislu1-1/+1
follow up with https://github.com/seaweedfs/seaweedfs/pull/5819
2024-07-26Check `ShouldGrowVolumes` before returning error in assign. (#5819)Ruoxi1-1/+1
2024-07-24Fix dead lock (#5815)Chris Lu9-37/+15
* reduce locks to avoid dead lock Flush->FlushData->uplloadPipeline.FluahAll uploaderCount>0 goroutine 1 [sync.Cond.Wait, 71 minutes]: sync.runtime_notifyListWait(0xc0007ae4d0, 0x0) /usr/local/go/src/runtime/sema.go:569 +0x159 sync.(*Cond).Wait(0xc001a59290?) /usr/local/go/src/sync/cond.go:70 +0x85 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).waitForCurrentWritersToComplete(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline_lock.go:58 +0x32 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).FlushAll(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline.go:151 +0x25 github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).FlushData(0xc00087e840) /github/workspace/weed/mount/dirty_pages_chunked.go:54 +0x29 github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).FlushData(...) /github/workspace/weed/mount/page_writer.go:50 github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).doFlush(0xc0006ad600, 0xc00030d380, 0x0, 0x0) /github/workspace/weed/mount/weedfs_file_sync.go:101 +0x169 github.com/seaweedfs/seaweedfs/weed/mount.(*WFS).Flush(0xc0006ad600, 0xc001a594a8?, 0xc0004c1ca0) /github/workspace/weed/mount/weedfs_file_sync.go:59 +0x48 github.com/hanwen/go-fuse/v2/fuse.doFlush(0xc0000da870?, 0xc0004c1b08) SaveContent -> MemChunk.RLock -> ChunkedDirtyPages.saveChunkedFileIntervalToStorage pages.fh.AddChunks([]*filer_pb.FileChunk{chunk}) fh.entryLock.Lock() sync.(*RWMutex).Lock(0x0?) /usr/local/go/src/sync/rwmutex.go:146 +0x31 github.com/seaweedfs/seaweedfs/weed/mount.(*FileHandle).AddChunks(0xc00030d380, {0xc00028bdc8, 0x1, 0x1}) /github/workspace/weed/mount/filehandle.go:93 +0x45 github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).saveChunkedFileIntervalToStorage(0xc00087e840, {0x2be7ac0, 0xc00018d9e0}, 0x0, 0x121, 0x17e3c624565ace45, 0x1?) /github/workspace/weed/mount/dirty_pages_chunked.go:80 +0x2d4 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).SaveContent(0xc0008d9130, 0xc0008093e0) /github/workspace/weed/mount/page_writer/page_chunk_mem.go:115 +0x112 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).moveToSealed.func1() /github/workspace/weed/mount/page_writer/upload_pipeline.go:187 +0x55 github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute.func1() /github/workspace/weed/util/limited_executor.go:38 +0x62 created by github.com/seaweedfs/seaweedfs/weed/util.(*LimitedConcurrentExecutor).Execute in goroutine 1 /github/workspace/weed/util/limited_executor.go:33 +0x97 On metadata update fh.entryLock.Lock() fh.dirtyPages.Destroy() up.chunksLock.Lock => each sealed chunk.FreeReference => MemChunk.Lock goroutine 134 [sync.RWMutex.Lock, 71 minutes]: sync.runtime_SemacquireRWMutex(0xc0007c3558?, 0xea?, 0x3fb0800?) /usr/local/go/src/runtime/sema.go:87 +0x25 sync.(*RWMutex).Lock(0xc0007c35a8?) /usr/local/go/src/sync/rwmutex.go:151 +0x6a github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*MemChunk).FreeResource(0xc0008d9130) /github/workspace/weed/mount/page_writer/page_chunk_mem.go:38 +0x2a github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*SealedChunk).FreeReference(0xc00071cdb0, {0xc0006ba1a0, 0x20}) /github/workspace/weed/mount/page_writer/upload_pipeline.go:38 +0xb7 github.com/seaweedfs/seaweedfs/weed/mount/page_writer.(*UploadPipeline).Shutdown(0xc0002ee4d0) /github/workspace/weed/mount/page_writer/upload_pipeline.go:220 +0x185 github.com/seaweedfs/seaweedfs/weed/mount.(*ChunkedDirtyPages).Destroy(0xc0008cea40?) /github/workspace/weed/mount/dirty_pages_chunked.go:87 +0x17 github.com/seaweedfs/seaweedfs/weed/mount.(*PageWriter).Destroy(...) /github/workspace/weed/mount/page_writer.go:78 github.com/seaweedfs/seaweedfs/weed/mount.NewSeaweedFileSystem.func3({0xc00069a6c0, 0x30}, 0x6?) /github/workspace/weed/mount/weedfs.go:119 +0x17a github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.NewMetaCache.func1({0xc00069a6c0?, 0xc00069a480?}, 0x4015b40?) /github/workspace/weed/mount/meta_cache/meta_cache.go:37 +0x1c github.com/seaweedfs/seaweedfs/weed/mount/meta_cache.SubscribeMetaEvents.func1(0xc000661810) /github/workspace/weed/mount/meta_cache/meta_cache_subscribe.go:43 +0x570 * use locked entry everywhere * modifiable remote entry * skip locking after getting lock from fhLockTable
2024-07-213.713.71chrislu1-1/+1
2024-07-21avoid nilchrislu1-0/+1
2024-07-213.70chrislu1-1/+1
2024-07-21add version to filerchrislu4-322/+349
2024-07-20Revert "add collection for buckets"chrislu2-14/+3
This reverts commit 96af5712195be37b309115795066f17c7cc6126d.
2024-07-20get, set and add path confchrislu3-3/+25
2024-07-17add collection for bucketschrislu2-3/+14
2024-07-17refactoringchrislu2-45/+1
2024-07-17Azure sink: avoid overwriting existing fileschrislu1-1/+12
2024-07-16Added tls for http clients (#5766)vadimartynov51-177/+588
* Added global http client * Added Do func for global http client * Changed the code to use the global http client * Fix http client in volume uploader * Fixed pkg name * Fixed http util funcs * Fixed http client for bench_filer_upload * Fixed http client for stress_filer_upload * Fixed http client for filer_server_handlers_proxy * Fixed http client for command_fs_merge_volumes * Fixed http client for command_fs_merge_volumes and command_volume_fsck * Fixed http client for s3api_server * Added init global client for main funcs * Rename global_client to client * Changed: - fixed NewHttpClient; - added CheckIsHttpsClientEnabled func - updated security.toml in scaffold * Reduce the visibility of some functions in the util/http/client pkg * Added the loadSecurityConfig function * Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16[filer.sync] skip overwriting existing fresh entrychrislu1-0/+4
2024-07-16minorchrislu1-3/+3
2024-07-16Added loadSecurityConfigOnce (#5792)vadimartynov25-24/+31
2024-07-16[master] Do Automatic Volume Grow in background (#5781)Konstantin Lebedev8-43/+93
* Do Automatic Volume Grow in backgound * pass lastGrowCount to master * fix build * fix count to uint64
2024-07-16Fix mmap write fail (#5791)wusong1-1/+4
* [mount] fix GetAttr blocks count Signed-off-by: wang wusong <wangwusong@virtaitech.com> * [mount] fix mmap read Signed-off-by: wang wusong <wangwusong@virtaitech.com> --------- Signed-off-by: wang wusong <wangwusong@virtaitech.com> Co-authored-by: wang wusong <wangwusong@virtaitech.com>
2024-07-15Add warning with hints to why a EC shard cant be moved (#5790)Dan1-2/+6
2024-07-15[master] refactor HasGrowRequest to atomic bool (#5782)Konstantin Lebedev1-4/+4
refactor HasGrowRequest to atomit bool
2024-07-12metrics stats of volume layout depends on the data center (#5775)Konstantin Lebedev2-4/+4
stats volume layout depends on the data center
2024-07-12fix: avoid timeout if datacenter does not exist in topology (#5772)Konstantin Lebedev5-0/+31
* fix: avoid timeout if datacenter does not exist in topology * fix: error msg * fix: rm dublicate check * fix: compare * revert minor change
2024-07-12ensure metadata follow a specific folderchrislu6-6/+33
fix https://github.com/seaweedfs/seaweedfs/issues/5774
2024-07-12-metricsIp cmd flag (#5773)vadimartynov5-5/+48
* Added/Updated: - Added metrics ip options for all servers; - Fixed a bug with the selection of the binIp or ip parameter for the metrics handler; * Fixed cmd flags
2024-07-12Fix "no more writable volumes" error when volume grows and master leader ↵小羽1-3/+4
changed at the same time (#5771)
2024-07-12Different disk labels should not use the same DiskUsages instance while ↵小羽1-1/+1
master received volume heatbeat (#5770)
2024-07-11small refactoringchrislu2-8/+3
2024-07-11Added tls for http handlers (#5764)vadimartynov2-13/+79
* Added https handler for filer * Added example for security.toml
2024-07-11fix testchrislu1-3/+3
2024-07-11fix heap initchrislu1-8/+10
2024-07-11clean upchrislu1-3/+3
2024-07-11upgrade to github.com/redis/go-redis/v9chrislu25-31/+31
fix https://github.com/seaweedfs/seaweedfs/issues/4532
2024-07-11Squashed commit of the following:chrislu3-90/+383
commit 465b47b8f72668dfc12eaa72befb232513c9e1db Merge: f39272b54 9c440d472 Author: chrislu <chris.lu@gmail.com> Date: Wed Jul 10 23:56:51 2024 -0700 Merge branch 'master' into merged-persisted-logs commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89 Author: chrislu <chris.lu@gmail.com> Date: Wed Jul 10 23:49:26 2024 -0700 merge meta logs from multiple files
2024-07-10[s3] revert skip deletion error, since the error file was not found is ↵Konstantin Lebedev1-5/+7
already skipped on the side of the grpc function (#5760) * revert skip deletion error, since the error file was not found is already skipped on the side of the grpc function * fix response error * fix test_lifecycle_get * Revert "fix test_lifecycle_get" This reverts commit 8f991bdcf93d9a13c7787988173713ad1a263bae.
2024-07-10always copy ecj fileschrislu1-1/+1
2024-07-10Detect underflow when calculating unused space (#5758)Dan2-2/+84
* Detect underflow when calculating unused space * Detect underflow when calculating unused space
2024-07-08simplifychrislu1-5/+2
2024-07-08Fix S3 deletion in deep folders, and names with empty spaceschrislu1-2/+3
fix https://github.com/seaweedfs/seaweedfs/issues/5748
2024-07-07add a test casechrislu1-0/+12
2024-07-07skip checking if it is the root directorychrislu1-1/+1
2024-07-07add a simple testchrislu1-0/+22
2024-07-04add s3test for sql (#5718)Konstantin Lebedev10-366/+949
* add s3test for sql * fix test test_bucket_listv2_delimiter_basic for s3 * fix action s3tests * regen s3 api xsd * rm minor s3 test test_bucket_listv2_fetchowner_defaultempty * add docs * without xmlns
2024-07-03fix breadcrumbchrislu2-0/+77
2024-07-03volume.tier.upload: Fix deleting replicated volumes (#5743)rhysm1-1/+4
Co-authored-by: Rhys M <rhysm@protonmail.com>
2024-07-03log error messagechrislu1-2/+2
2024-07-02Revert "directory can paginate"chrislu2-11/+4
This reverts commit 69003d0e906e098cf6db11da05d915d0718a5eba.
2024-07-02fix orderingchrislu1-1/+1
2024-07-02directory can paginatechrislu2-4/+11
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-26 13:32:13 +0000