aboutsummaryrefslogtreecommitdiff
path: root/weed/util
AgeCommit message (Collapse)AuthorFilesLines
2024-09-14fix file read crash (#6021)Bruce1-4/+7
2024-09-123.733.73chrislu1-1/+1
2024-09-11Feature limit caching to prescribed number of bytes per file (#6009)Eugeniy E. Mikhailov1-0/+46
* feature: we can check if a fileId is already in the cache We using this to protect cache from adding the same needle to the cache over and over. * fuse mount: Do not start dowloader if needle is already in the cache * added maxFilePartSizeInCache property to ChunkCache If file very large only first maxFilePartSizeInCache bytes are going to be put to the cache (subject to the needle size constrains). * feature: for large files put in cache no more than prescribed number of bytes Before this patch only the first needle of a large file was intended for caching. This patch uses maximum prescribed amount of bytes to be put in cache. This allows to bypass default 2MB maximum for a file part stored in the cache. * added dummy mock methods to satisfy interfaces of ChunkCache
2024-09-10bug fix in the data received from cache processing (#6002)Eugeniy E. Mikhailov1-5/+5
The patch addresses #3745. The cache should return the exact amount of data requested by the buffer. By construction of the cache it is always all requested data range or we have error happening. The old use of minsize miscalculate the requested data size, if non zero offset is requested.
2024-09-04Revert "weed mount, weed dav add option to force cache"chrislu2-6/+4
This reverts commit 7367b976b05bfa69158a60f205dec970c48f50f0.
2024-09-04weed mount, weed dav add option to force cachechrislu2-4/+6
2024-08-183.723.72chrislu1-1/+1
2024-08-07randomizing next file handle idchrislu1-0/+6
2024-07-213.713.71chrislu1-1/+1
2024-07-213.70chrislu1-1/+1
2024-07-21add version to filerchrislu1-1/+3
2024-07-16Added tls for http clients (#5766)vadimartynov7-37/+319
* Added global http client * Added Do func for global http client * Changed the code to use the global http client * Fix http client in volume uploader * Fixed pkg name * Fixed http util funcs * Fixed http client for bench_filer_upload * Fixed http client for stress_filer_upload * Fixed http client for filer_server_handlers_proxy * Fixed http client for command_fs_merge_volumes * Fixed http client for command_fs_merge_volumes and command_volume_fsck * Fixed http client for s3api_server * Added init global client for main funcs * Rename global_client to client * Changed: - fixed NewHttpClient; - added CheckIsHttpsClientEnabled func - updated security.toml in scaffold * Reduce the visibility of some functions in the util/http/client pkg * Added the loadSecurityConfig function * Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-16Added loadSecurityConfigOnce (#5792)vadimartynov1-0/+7
2024-07-11Squashed commit of the following:chrislu1-0/+11
commit 465b47b8f72668dfc12eaa72befb232513c9e1db Merge: f39272b54 9c440d472 Author: chrislu <chris.lu@gmail.com> Date: Wed Jul 10 23:56:51 2024 -0700 Merge branch 'master' into merged-persisted-logs commit f39272b5416bdd4d1714ce3e99aded0fe62e7d89 Author: chrislu <chris.lu@gmail.com> Date: Wed Jul 10 23:49:26 2024 -0700 merge meta logs from multiple files
2024-07-07add a simple testchrislu1-0/+22
2024-07-01refactor all methods strings to const (#5726)Konstantin Lebedev1-7/+7
2024-06-303.69chrislu1-1/+1
2024-06-28bootstrap filer from one peerchrislu1-14/+16
2024-06-093.683.68chrislu1-1/+1
2024-05-20go fmtchrislu1-2/+2
2024-05-073.673.67chrislu1-1/+1
2024-05-07Fix deadlock in lock table locks (#5566)Patrick Schmidt1-6/+13
2024-05-053.663.66chrislu1-1/+1
2024-05-05ensure head index is within rangechrislu1-2/+8
2024-04-25Merge branch 'master' into mq-subscribechrislu2-2/+10
2024-04-25Fixes unlocked read from logBuffer.LastTsNs that is racey. (#5536)M@1-1/+9
2024-04-24filer read chunk retry if status code 499 (#5528)Konstantin Lebedev1-1/+1
2024-04-15Merge branch 'master' into mq-subscribechrislu1-1/+1
2024-04-143.653.65chrislu1-1/+1
2024-04-11Merge branch 'master' into mq-subscribechrislu1-0/+20
2024-04-11[volume] Reduce the number of buffers for uploading one chunk (#5458)Konstantin Lebedev1-0/+20
2024-04-08Merge branch 'master' into mq-subscribechrislu2-16/+23
2024-04-08fix: filer authenticate with with volume server (#5480)Henco Appel1-15/+22
2024-04-08chore: fix function names in comment (#5478)clonefetch1-1/+1
2024-04-02refactorchrislu2-2/+12
2024-04-01shutdown follower when leader stopschrislu1-0/+13
2024-04-01send flush message to follower before shutting down logBufferchrislu1-0/+8
2024-03-21clean up logschrislu1-1/+1
2024-03-21adjust log levelchrislu1-1/+1
2024-03-21clean up logschrislu1-2/+2
2024-03-21adjust log levelchrislu1-1/+1
2024-03-21adjust log levelchrislu1-1/+1
2024-03-16fix test2chrislu1-5/+5
2024-03-163.643.64chrislu1-1/+1
2024-03-16wait for datachrislu1-5/+9
2024-03-16rename to LastTsNschrislu1-4/+4
2024-03-16go fmtchrislu1-15/+15
2024-03-16rename to lastFlushDataTimechrislu1-7/+7
2024-03-10ask follower to followchrislu2-1/+7
2024-03-07include key in LogEntrychrislu1-0/+1