Commit Graph

771 Commits

Author SHA1 Message Date
Zhe Wang a3abfad67a
reduce ShouldCheckPeer frequency and increase max_trace_lines for bulkload tests (#11935) 2025-02-10 17:59:03 -08:00
Syed Paymaan Raza 26f38c8970
Gray failure observability (#11923) 2025-02-06 17:16:14 -08:00
Jingyu Zhou df0fc7d3b8
Fix issues with clang 19 (#11834)
* Fix issues with clang 19

* Fix format

* Ignore --undefined-version for gcc
2024-12-13 14:20:45 -08:00
Zhe Wang f6be4a4b94 bulkdump generate global manifest 2024-12-03 22:01:07 -08:00
Syed Paymaan Raza 7a5f61cc65
Address feedback in PR #11753 (#11757) 2024-11-05 20:30:40 -08:00
Syed Paymaan Raza 84fb8f843c
Gray failure allows storage servers to complain (#11753) 2024-11-05 16:53:02 -08:00
Syed Paymaan Raza c146ee0869
[fdbserver] Use STL contains method and std::find for containment checks (#11702) 2024-10-15 11:40:02 -07:00
Jingyu Zhou 80ca71833b Make xxhash checksum the default for TLog
Update downgrade tests to use the xxhash.
2024-09-17 12:46:42 -07:00
Jingyu Zhou 43dd7595c5 Remove old tlog 4.6, 6.0, and 6.2
These are unsupported versions now.
2024-09-17 12:46:42 -07:00
Syed Paymaan Raza 48064f6cf1
Make some codeprobes rare (#11607)
* Make BlobGranule code probes rare

* Make encryption related code probes rare

* fixup! Fix formatting
2024-08-26 22:33:38 -07:00
Syed Paymaan Raza c3e7542cda Update end year in copyright header 2024-08-02 09:40:11 -07:00
Zhe Wang 0b3bc71689
Fix TraceTooManyLines in BulkLoading Simulation Test (#11531)
* fix-trace-too-many-lines-bulkload-test

* mute PeerHealthMonitor

* address comments
2024-07-26 17:47:34 -07:00
Jingyu Zhou a733e48048
Merge pull request #11517 from jzhou77/release-notes
Fix Wunused-but-set-variable warnings
2024-07-18 13:51:21 -07:00
Jingyu Zhou 078aaf153c
Merge pull request #11497 from xis19/4846
Add PeerAddress to all PeerAddr/Peer TraceEvent
2024-07-18 10:09:57 -07:00
Jingyu Zhou d9e4c49503 Fix more -Wunused-variable warnings 2024-07-17 15:35:49 -07:00
Xiaoge Su cf70d45e6d Add PeerAddress to all PeerAddr/Peer TraceEvent
This is to address #4846
2024-07-09 16:49:22 -07:00
Zhe Wang e9505506df
add consistency-check-urgent-mode to tester process class (#11484) 2024-07-09 13:21:37 -07:00
Dimitris Apostolou a88114c222
Fix typos 2024-02-07 01:16:00 +02:00
Jingyu Zhou f72eab118d Skip redwood for clearInflightCommits unit test
Redwood complains DB is invalid for this unit test:

Assertion keyProvider.isValid() || db.isValid() failed @ /root/src/foundationdb/fdbserver/VersionedBTree.actor.cpp 8029:
2023-09-18 14:34:36 -07:00
Jingyu Zhou 977851fa39 Fix sharded rocks failure by adding a shard 2023-09-13 15:53:27 -07:00
Jingyu Zhou 42df53b3bc Add a test case for storage engine
This test validates that in-flight commit to the storage engine is properly
handled. As found in https://github.com/apple/foundationdb/pull/10714, an
engine could misses in-flight data and cause data corruptions.

The test case is modeled after the above corruption: insert data, then clear
the data in the next commit to the storage engine, and finally varify that the
data is cleared.
2023-09-13 15:25:25 -07:00
Yi Wu 3287098b4a EaR: Handle KMS timeout in storage server and commit proxy 2023-08-28 16:17:43 -07:00
Zhe Wang f8311ae069
Add more trace event for TSS recruitment (#10809)
* add more trace event for tss

* update StorageServerInitProgress

* add more traces
2023-08-23 09:19:30 -07:00
Zhe Wang 83dc9ff6f7
Trace SS init progress (#10799)
* trace ss init progress

* improve trace events
2023-08-18 18:44:37 -07:00
Zhe Wu 2078a0055a Add documentation 2023-07-26 14:16:16 -07:00
Zhe Wu f9c3ac3704 Remove debugging logging 2023-07-26 12:35:19 -07:00
Zhe Wu e583deddfe Apply format 2023-07-26 11:21:20 -07:00
Zhe Wu 00cdf09966 Cluster controller monitors recovery stats and trigger recovery if current recovery contains degraded servers 2023-07-17 11:14:22 -07:00
Zhe Wu 9670ed1cd8 Make TLog explicitely monitor connectivity issue during [start version, recover version] recovery 2023-07-17 11:12:28 -07:00
Zhe Wu 458d210584 Add an option in worker to force detect connectivity issue to other transaction workers in primary region 2023-07-17 11:12:28 -07:00
Yi Wu 92fc089eb0
FDBCORE-5211: Fix worker server handleIOErrors heap-use-after-free (#380) (#10555) 2023-06-26 16:35:33 -07:00
Zhe Wu 36befca2ba Nit fixing 2023-06-21 09:25:51 -07:00
Zhe Wu f3e95f6698
Merge pull request #10521 from halfprice/zhewu/gray-failure-recovery
Worker health monitor tracks recovered peers
2023-06-21 07:56:55 -07:00
Zhe Wu 31de83ccc1 cleanup gray failure recovery documentation 2023-06-20 13:30:03 -07:00
Zhe Wu 4741d7032f Revert logic to test testing logic 2023-06-20 12:40:22 -07:00
Zhe Wu ef638cd6a6 Worker health monitor keeps track of recovered peer 2023-06-20 12:32:29 -07:00
Evan Tschannen ef682d304e fix IKeyValueStore include 2023-06-16 13:28:40 -07:00
Evan Tschannen eb772c0043 added a blob worker specific page cache size for redwood so that it does not have to be changed manually in fdb.conf for all blob worker processes 2023-06-13 10:35:13 -07:00
Evan Tschannen 359e178dcd Merge branch 'main' into feature-durable-change-feed
# Conflicts:
#	fdbclient/ClientKnobs.cpp
#	fdbserver/BlobManager.actor.cpp
#	fdbserver/worker.actor.cpp
2023-06-11 13:58:35 -07:00
Evan Tschannen 2a8e5e487d fix: properly dispose of unneeded blob worker disk stores 2023-06-10 22:09:30 -07:00
Evan Tschannen b15738ad8f do not start a blob worker from disk is the feature is disabled 2023-06-09 14:08:02 -07:00
Evan Tschannen 197c39b552 cache change feeds using a storage engine to avoid reading them for the server on startup 2023-06-07 08:41:31 -07:00
Dan Adkins 833bcb9342 Let coordination server crash on file_not_found error.
The only case when a coordination server should fail with file_not_found
is when the DiskQueue only finds 1 out of its 2 queue files. This either
means data has been lost, or the initial creation of the two files failed
partway through. In either case, it's sensible to crash the process and
refuse to start.

The existing code was catching the file_not_found error and letting the
FDBD process continue after the coordination server failed. This resulted
in clients attempting to contact the coordination server on a well-known
endpoint and never getting a response, because Flow transport silently
dropped the request when there was no receiver for it. By terminating the
process instead, we effectively notify waiting clients that they won't
get a response from this coordinator.
2023-06-06 11:36:25 -07:00
Yao Xiao cefb6c7ec3
Clear metadata before destroyDB. (#10393) 2023-06-02 11:20:14 -07:00
Nim Wijetunga 95bf14323f
EKP and KMS Health Check (#10341)
EKP and KMS Health Check
2023-06-01 16:24:04 -07:00
Zhe Wu 2e00e0abed Refactor healthMonitor in worker.actor.cpp 2023-05-23 14:14:33 -07:00
Evan Tschannen 3dd86d6c22 move IKeyValueStore.h to the client 2023-05-10 15:41:47 -07:00
Yao Xiao abd45c4486 Cherrypick #9665 2023-05-10 10:23:01 -07:00
Chaoguang Lin e318fc2600
Fix for correctness failures when issuing duplicate snapshot requests (#10041)
* Fix for correctness failures when issuing duplicate requests
Add comments; Disable failure injection in Snap test

* Format files
2023-04-25 13:48:50 -07:00
Zhe Wu 10a6f3d2d0
Merge pull request #9890 from halfprice/zhewu/log-router-gray-failure
Gray failure detects disconnected remote log router and recover high DC lag
2023-04-07 16:25:11 -07:00