From 084b377f8786e3a4d98e0763c3e83be104a9b65e Mon Sep 17 00:00:00 2001
From: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.com>
Date: Thu, 6 Nov 2025 11:05:54 +0500
Subject: do delete expired entries on s3 list request (#7426)

* do delete expired entries on s3 list request
https://github.com/seaweedfs/seaweedfs/issues/6837

* disable delete expires s3 entry in filer

* pass opt allowDeleteObjectsByTTL to all servers

* delete on get and head

* add lifecycle expiration s3 tests

* fix opt allowDeleteObjectsByTTL for server

* fix test lifecycle expiration

* fix IsExpired

* fix locationPrefix for updateEntriesTTL

* fix s3tests

* resolv  coderabbitai

* GetS3ExpireTime on filer

* go mod

* clear TtlSeconds for volume

* move s3 delete expired entry to filer

* filer delete meta and data

* del unusing func removeExpiredObject

* test s3 put

* test s3 put multipart

* allowDeleteObjectsByTTL by default

* fix pipline tests

* rm dublicate SeaweedFSExpiresS3

* revert expiration tests

* fix updateTTL

* rm log

* resolv comment

* fix delete version object

* fix S3Versioning

* fix delete on FindEntry

* fix delete chunks

* fix sqlite not support concurrent writes/reads

* move deletion out of listing transaction; delete entries and empty folders

* Revert "fix sqlite not support concurrent writes/reads"

This reverts commit 5d5da14e0ed91c613fe5c0ed058f58bb04fba6f0.

* clearer handling on recursive empty directory deletion

* handle listing errors

* strut copying

* reuse code to delete empty folders

* use iterative approach with a queue to avoid recursive WithFilerClient calls

* stop a gRPC stream from the client-side callback is to return a specific error, e.g., io.EOF

* still issue UpdateEntry when the flag must be added

* errors join

* join path

* cleaner

* add context, sort directories by depth (deepest first) to avoid redundant checks

* batched operation, refactoring

* prevent deleting bucket

* constant

* reuse code

* more logging

* refactoring

* s3 TTL time

* Safety check

---------

Co-authored-by: chrislu <chris.lu@gmail.com>
---
 weed/util/constants_lifecycle_interval_10sec.go | 8 ++++++++
 weed/util/constants_lifecycle_interval_day.go   | 8 ++++++++
 2 files changed, 16 insertions(+)
 create mode 100644 weed/util/constants_lifecycle_interval_10sec.go
 create mode 100644 weed/util/constants_lifecycle_interval_day.go

(limited to 'weed/util')

diff --git a/weed/util/constants_lifecycle_interval_10sec.go b/weed/util/constants_lifecycle_interval_10sec.go
new file mode 100644
index 000000000..60f19c316
--- /dev/null
+++ b/weed/util/constants_lifecycle_interval_10sec.go
@@ -0,0 +1,8 @@
+//go:build s3tests
+// +build s3tests
+
+package util
+
+import "time"
+
+const LifeCycleInterval = 10 * time.Second
diff --git a/weed/util/constants_lifecycle_interval_day.go b/weed/util/constants_lifecycle_interval_day.go
new file mode 100644
index 000000000..e2465ad5f
--- /dev/null
+++ b/weed/util/constants_lifecycle_interval_day.go
@@ -0,0 +1,8 @@
+//go:build !s3tests
+// +build !s3tests
+
+package util
+
+import "time"
+
+const LifeCycleInterval = 24 * time.Hour
-- 
cgit v1.2.3