aboutsummaryrefslogtreecommitdiff
path: root/weed/operation
AgeCommit message (Collapse)AuthorFilesLines
2025-06-20Context-based logging with request ID (#6899)Aleksey Kosov1-2/+4
2025-06-16add default value when reading needle versionchrislu1-0/+3
2025-06-16tail-volume-uses-the-source-volume-versionchrislu1-1/+2
2025-06-16refactoringchrislu1-2/+3
2025-05-28Add context with request (#6824)Aleksey Kosov5-25/+29
2024-12-19"golang.org/x/exp/slices" => "slices" and go fmtchrislu1-6/+6
2024-11-15Fix/parse upload filename (#6241)Kevin Bulteel1-1/+1
* fix: parse filename in PUT + refactor * fix: master iu public url with http * fix: better parsing and handle disposition header * fix: take mime type from file extension if not set
2024-11-04merge current message queue code changes (#6201)Chris Lu1-39/+41
* listing files to convert to parquet * write parquet files * save logs into parquet files * pass by value * compact logs into parquet format * can skip existing files * refactor * refactor * fix compilation * when no partition found * refactor * add untested parquet file read * rename package * refactor * rename files * remove unused * add merged log read func * parquet wants to know the file size * rewind by time * pass in stop ts * add stop ts * adjust log * minor * adjust log * skip .parquet files when reading message logs * skip non message files * Update subscriber_record.go * send messages * skip message data with only ts * skip non log files * update parquet-go package * ensure a valid record type * add new field to a record type * Update read_parquet_to_log.go * fix parquet file name generation * separating reading parquet and logs * add key field * add skipped logs * use in memory cache * refactor * refactor * refactor * refactor, and change compact log * refactor * rename * refactor * fix format * prefix v to version directory
2024-09-15Feature: compress before encryption (#6013)Eugeniy E. Mikhailov1-2/+5
2024-09-09[shell] fix volume grow in shell (#5992)Konstantin Lebedev1-1/+0
* fix volume grow in shell * revert add Async * check available volume space * create a VolumeGrowRequest and remove unnecessary fields
2024-08-29math/rand => math/rand/v2chrislu2-4/+4
2024-08-27randomly pick one replica to writechrislu1-10/+24
2024-08-01rename functionschrislu2-8/+8
2024-07-16Added tls for http clients (#5766)vadimartynov4-48/+81
* Added global http client * Added Do func for global http client * Changed the code to use the global http client * Fix http client in volume uploader * Fixed pkg name * Fixed http util funcs * Fixed http client for bench_filer_upload * Fixed http client for stress_filer_upload * Fixed http client for filer_server_handlers_proxy * Fixed http client for command_fs_merge_volumes * Fixed http client for command_fs_merge_volumes and command_volume_fsck * Fixed http client for s3api_server * Added init global client for main funcs * Rename global_client to client * Changed: - fixed NewHttpClient; - added CheckIsHttpsClientEnabled func - updated security.toml in scaffold * Reduce the visibility of some functions in the util/http/client pkg * Added the loadSecurityConfig function * Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-01refactor all methods strings to const (#5726)Konstantin Lebedev2-2/+2
2024-06-14Added context for the MasterClient's methods to avoid endless loops (#5628)vadimartynov5-9/+12
* Added context for the MasterClient's methods to avoid endless loops * Returned WithClient function. Added WithClientCustomGetMaster function * Hid unused ctx arguments * Using a common context for the KeepConnectedToMaster and WaitUntilConnected functions * Changed the context termination check in the tryConnectToMaster function * Added a child context to the tryConnectToMaster function * Added a common context for KeepConnectedToMaster and WaitUntilConnected functions in benchmark
2024-04-11[volume] Reduce the number of buffers for uploading one chunk (#5458)Konstantin Lebedev1-5/+20
2024-03-29[master] grow volumes if no writable volumes in current dataCenter (#5434)Konstantin Lebedev1-0/+3
* grow volumes if no writable volumes in current dataCenter https://github.com/seaweedfs/seaweedfs/issues/3886 * fix tests with volume grow * automatic volume grow one volume * add ErrorChunkAssign metrics
2024-01-10chore: add status code for request_total metrics (#5188)Konstantin Lebedev1-1/+1
2023-10-12 fix: disallow file name too long when writing a file (#4881)Konstantin Lebedev1-1/+2
* fix: disallow file name too long when writing a file * bool LongerName to MaxFilenameLength --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-10-03weed/operation: fix dropped error (#4888)Lars Lehtonen1-0/+3
2023-10-01RetryForever => RetryUntilchrislu1-1/+1
2023-09-10ensure single threaded accesschrislu1-0/+5
2023-08-23streaming assign file idschrislu2-3/+170
2023-04-29[Filer] post add param:saveInside (#4434)zemul1-0/+1
* fix:mount deadlock * feat: filer http upload to metadata * feat: /etc save inside --------- Co-authored-by: zemul <zhouzemiao@ihuman.com>
2023-04-21mount: add retry for read only case (#4416)wusong1-6/+8
* mount: add retry for read only case Signed-off-by: Wusong Wang <wangwusong@virtaitech.com> * add new util retry function for mount Signed-off-by: Wusong Wang <wangwusong@virtaitech.com> * change error list param Signed-off-by: Wusong Wang <wangwusong@virtaitech.com> --------- Signed-off-by: Wusong Wang <wangwusong@virtaitech.com> Co-authored-by: Wusong Wang <wangwusong@virtaitech.com>
2023-01-20grpc connection to filer add sw-client-id headerchrislu1-2/+2
2023-01-02more solid weed mount (#4089)Chris Lu1-2/+2
* compare chunks by timestamp * fix slab clearing error * fix test compilation * move oldest chunk to sealed, instead of by fullness * lock on fh.entryViewCache * remove verbose logs * revert slat clearing * less logs * less logs * track write and read by timestamp * remove useless logic * add entry lock on file handle release * use mem chunk only, swap file chunk has problems * comment out code that maybe used later * add debug mode to compare data read and write * more efficient readResolvedChunks with linked list * small optimization * fix test compilation * minor fix on writer * add SeparateGarbageChunks * group chunks into sections * turn off debug mode * fix tests * fix tests * tmp enable swap file chunk * Revert "tmp enable swap file chunk" This reverts commit 985137ec472924e4815f258189f6ca9f2168a0a7. * simple refactoring * simple refactoring * do not re-use swap file chunk. Sealed chunks should not be re-used. * comment out debugging facilities * either mem chunk or swap file chunk is fine now * remove orderedMutex as *semaphore.Weighted not found impactful * optimize size calculation for changing large files * optimize performance to avoid going through the long list of chunks * still problems with swap file chunk * rename * tiny optimization * swap file chunk save only successfully read data * fix * enable both mem and swap file chunk * resolve chunks with range * rename * fix chunk interval list * also change file handle chunk group when adding chunks * pick in-active chunk with time-decayed counter * fix compilation * avoid nil with empty fh.entry * refactoring * rename * rename * refactor visible intervals to *list.List * refactor chunkViews to *list.List * add IntervalList for generic interval list * change visible interval to use IntervalList in generics * cahnge chunkViews to *IntervalList[*ChunkView] * use NewFileChunkSection to create * rename variables * refactor * fix renaming leftover * renaming * renaming * add insert interval * interval list adds lock * incrementally add chunks to readers Fixes: 1. set start and stop offset for the value object 2. clone the value object 3. use pointer instead of copy-by-value when passing to interval.Value 4. use insert interval since adding chunk could be out of order * fix tests compilation * fix tests compilation
2022-10-28refactor filer proto chunk variable from mtime to modified_ts_nschrislu1-1/+1
2022-10-19add md5 header when UploadData to replication in ReplicatedWrite (#3881)liubaojiang1-0/+5
2022-09-10refactorchrislu1-3/+4
2022-08-31upload_content: upload close response.Bodychrislu1-1/+2
2022-08-31refactorchrislu1-4/+1
2022-08-23remove old raft servers if they don't answer to pings for too long (#3398)askeipx1-2/+2
* remove old raft servers if they don't answer to pings for too long add ping durations as options rename ping fields fix some todos get masters through masterclient raft remove server from leader use raft servers to ping them CheckMastersAlive for hashicorp raft only * prepare blocking ping * pass waitForReady as param * pass waitForReady through all functions * waitForReady works * refactor * remove unneeded params * rollback unneeded changes * fix
2022-08-20adjust logchrislu1-1/+1
2022-08-20add fileId to UploadWithRetry return resultchrislu1-2/+2
2022-08-20filer.copy: retryable uploadchrislu1-0/+49
2022-08-04filer prefer volume server in same data center (#3405)Konstantin Lebedev2-9/+11
* initial prefer same data center https://github.com/seaweedfs/seaweedfs/issues/3404 * GetDataCenter * prefer same data center for ReplicationSource * GetDataCenterId * remove glog
2022-08-04refactor FilerRequest metrics (#3402)Konstantin Lebedev1-0/+2
* refactor FilerRequest metrics * avoid double count proxy * defer to
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu11-29/+29
2022-03-15reduced the timeout for creating a connection from 30 to 10 secondsKonstantin Lebedev1-0/+6
2022-02-15weed/operation/delete_content.go: nil pointer dereferencebanjiaojuhao1-1/+1
2022-01-05mount: skip special character in the filenameschrislu1-7/+7
fix https://github.com/chrislusf/seaweedfs/issues/2559
2021-12-26use streaming mode for long poll grpc callschrislu6-10/+10
streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed.
2021-12-22filer server: allow upload file to specific dataNodebanjiaojuhao1-1/+4
2021-10-14refactor: move from io/ioutil to io and os packageEng Zer Jun2-6/+4
The io/ioutil package has been deprecated as of Go 1.16, see https://golang.org/doc/go1.16#ioutil. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-10-11return error earlyChris Lu1-4/+4
fix https://github.com/chrislusf/seaweedfs/issues/2370
2021-09-14go fmtChris Lu1-1/+1
2021-09-12change server address from string to a typeChris Lu8-74/+46
2021-09-08replicated remote.cacheChris Lu1-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 00:06:49 +0000