Commit Graph

81 Commits

Author SHA1 Message Date
Chris Douglas fec0fb9e9f Move awscli to base image from YCSB 2021-12-17 11:25:22 -08:00
Aaron Molitor 95d33cb363 copy packaging/docker to PROJECT_BINARY_DIR (undoing part of #5994),
fetch commit_sha from source_code_directory (don't assume we're in the source tree anymore),
allow custom tag (if a parameter is passed in as $1)
update README.md
2021-12-15 15:23:17 -08:00
A.J. Beamon f24adc7b6a Fix a bunch of places where we used old-style arguments. Allow hyphens for profiler args. 2021-12-14 09:59:14 -08:00
A.J. Beamon f29f487823
Unify flags (#25)
* Unify flags implementation and change help text in backup.actor.cpp
Description

Testing

* Keep LOG_GROUP unchanged

Description

Testing

* Transfer the hyphens to underscores for internal options and user's input, EXCEPT leading hyphens

Description

Testing

* Use a deep copy of the user's input flag to do the match

Description

Testing

* Convert the _ to - in Option arrays of backup.actor.cpp

Description

Testing

* Transter _ to - for files:
        TLSConfig.actor.h, fdbcli.actor.cpp, fdbserver.actor.cpp, FileConverter.h, FileConverter.cpp

Description

Testing

* Change another way to unify flag: using SO_O_ICASE_HYPHEN_AND_UNDERSCORE to determine whether we do the conversion in function IsEqual

Description

Testing

* Change the config command's name from SO_O_ICASE_HYPHEN_AND_UNDERSCORE to SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Update the comment for the SO_O_HYPHEN_TO_UNDERSCORE

Description

Testing

* Fix left underscore in SOption arrays

Description

Testing

* Convert _ to - in several files for commands

Description

Testing

* Make the FDBService and fdbmonitor backward compatible

Description

Testing

* Fix bugs about pointers

Description

Testing

* Check underscore and hyphen at the same time for --knob_, --localily_ and --test_
And fix bugs in fdbmonitor and FDBService
Description

Testing

* Simplify the function in fdbmonitor and FDBService about retrieving arguments.
And fix some documents in masterserver.actor.cpp

Description

Testing

* Convert _ to - for knob in the setKnob functions

Description

Testing

* Convert - to _ in the setKnob functions

Description
Since key in the knob related maps only contain _

Testing

* Rename varialbe name in the fdbmonitor and FDBService for clarification

Description

Testing

Co-authored-by: Chang Liu <chang.liu@snowflake.com>
2021-12-14 08:44:39 -08:00
Aaron Molitor 8aab68303b add discrete tag_postfix to images, cleanup documentation 2021-12-10 15:37:04 -08:00
Aaron Molitor 661181d61c make curl and sha256sum more quiet, pull local binaries from the correct place in build_output_directory 2021-12-03 15:06:09 -08:00
Russell Sears d332d03b34 Have individual YCSB containers record completion of their phase of the load. This allows loads to succeed even if the fdb-kubernetes-tests client crashes out mid run 2021-11-19 14:33:23 -08:00
Aaron Molitor 124e7eb4f2 refactor docker directory
- make release images ALL based from centos:7
- keep eks images that are based from amazonlinux:2, but create strong alignment with release (centos7) images
- use multi-stage Dockerfile(s)
- have a single build-images script
- remove per image directories (which would only contain scripts that are copied into images)
- remove empty requirements.txt file for sidecar.py (the pip command, package and version are in the Dockerfile)
- don't copy docker into PROJECT_BINARY_DIR
- incorporate latest changes already in master
2021-11-17 16:29:07 -08:00
John Brownlee 01c37a053a
Merge pull request #5993 from brownleej/new-image-fixes
Updates to fdb-kubernetes-monitor to ease integration with the Kubernetes operator
2021-11-17 11:56:19 -08:00
John Brownlee 2f2e5219da Move the config file scheme into a separate package so it can be imported by the operator.
Move the argument for the number of servers per pod into a command-line argument so we can use the same config file at different storage server densities.
2021-11-12 11:58:38 -08:00
Yao Xiao 86ea63d1da
Add libsanitizer to fdb image. (#5918) 2021-11-04 23:21:54 -07:00
John Brownlee 45a66f9f75
Merge pull request #5456 from brownleej/fdb-kubernetes-monitor
Create fdb-kubernetes-monitor subproject
2021-10-29 11:29:23 -07:00
Aaron Molitor 504f08a102 consolidate docker stuff, add perf and flamegraph parts to release image 2021-10-29 09:58:55 -07:00
Leonidas Tsampros a4d784a3dc packaging: apt doesn't support >= and fix tini installation 2021-10-29 09:58:55 -07:00
Aaron Molitor 3305fe92f6 fix typo in script 2021-10-28 19:45:48 -07:00
John Brownlee a6b903e7f8 Move the new Kubernetes image to centos 7. 2021-10-28 17:40:05 -07:00
John Brownlee 1a5069a047 Use an write-and-rename pattern when copying files for atomicity.
Restructure the usage of the mutex in the monitor class.
2021-10-28 17:40:05 -07:00
John Brownlee c7858d2441 Add the init and sidecar modes for fdb-kubernetes-monitor.
Add support for using a special binary path during upgrades in fdb-kubernetes-monitor.
2021-10-28 17:40:05 -07:00
John Brownlee 7c36123cf8 Adds structured logging in fdb-kubernetes-monitor.
Adds a backoff window when restarting processes in fdb-kubernetes-monitor.
2021-10-28 17:40:05 -07:00
John Brownlee 95ad5854be Add a mechanism to post updates from fdb-kubernetes-monitor to pod annotations.
Remove some of the local test data for fdb-kubernetes-monitor in favor of testing through a Kuberentes statefulset.
2021-10-28 17:40:05 -07:00
John Brownlee 0f173edb47 Add a dockerfile for fdb-kubernetes-monitor. 2021-10-28 17:40:05 -07:00
Aaron Molitor e9091f91c9 make build-ycsb-docker.sh executalbe 2021-10-18 14:55:39 -07:00
Aaron Molitor 7129c2b528 update packaging to align across branches 2021-10-18 14:31:46 -07:00
Aaron Molitor 82bdd03268 add ycsb docker build script 2021-10-18 13:57:39 -07:00
Aaron Molitor 73525085cf cleanup docker build scripts 2021-10-18 13:57:39 -07:00
Aaron Molitor 0e4a6cc921 add ycsb Dockerfile 2021-10-15 00:27:20 -07:00
Johannes M. Scheuermann a597b87af9 Fix typo for None 2021-09-21 10:33:11 +01:00
Johannes M. Scheuermann e091951b8e Fix kubernetes sidecar 2021-09-21 10:33:11 +01:00
Johannes M. Scheuermann 33928904ec Update sidecar to match latest changes in operator 2021-09-21 10:33:11 +01:00
A.J. Beamon 94c3cb3086 Create the logs directory for FDB processes in the correct place 2021-08-27 10:58:32 -07:00
falsandtru 0527cccdab Fix Golang sample 2021-08-13 02:18:21 +09:00
falsandtru 5a846bddd5 Fix download addresses 2021-08-13 02:17:42 +09:00
Daniel Smith 2524a6acc3 Debugging help 2021-06-10 14:04:00 -04:00
Meng Xu 60a9dda0ed
Merge pull request #4899 from Daniel-B-Smith/aws-in-docker
Add AWS CLI tool for downloading from s3
2021-06-09 16:34:57 -07:00
Daniel Smith 948524b95e Add AWS CLI tool for downloading from s3 2021-06-03 12:08:14 -04:00
Russell Sears c80cd575ad Add perl + flamegraph to EKS docker images 2021-05-07 11:24:47 -07:00
Russell Sears 5fed5640d3
Merge pull request #4718 from sears/build-k8s-images
Build k8s images
2021-04-28 17:09:38 -07:00
Russell Sears 55ce798ff4 Reconcile differences in release and sidecar Dockerfile 2021-04-28 05:41:01 +00:00
Russell Sears 6b0ba14199 make release docker build script match EKS one as much as possible 2021-04-27 21:24:39 -07:00
Russell Sears 2eef4e28be Documentation update 2021-04-27 17:15:44 -07:00
Russell Sears ec0f5db98e port sidecar.py to distributions that default to python 2 instead of 3
fix various entrypoint bugs in sidecar
2021-04-27 16:57:35 -07:00
Russell Sears 1b5119b7fb Add new docker images for EKS environments 2021-04-27 12:24:54 -07:00
Russell Sears 8dc487e562 docker cleanup 2021-04-26 16:23:54 -07:00
Russell Sears f155374fd2 Copy sidecar docker from fdb-kubernetes-operator github (the intent is to delete it from the other repo) 2021-04-26 22:41:55 +00:00
Russell Sears c3ce091602 remove unused Dockerfiles 2021-04-26 15:21:07 -07:00
Russell Sears fabeedb527 improve docker packaging script 2021-04-26 15:20:25 -07:00
Russell Sears 5e5ccebb4c Draft scripts to build kubernetes image + sidecar. Depends on okteto default paths (that is a feature, not a bug). 2021-04-23 18:41:04 -07:00
john_leach b39b2b4a3c Initial Container structure, build images from build_output/packages/ 2021-04-23 15:33:01 -07:00
falsandtru 225375b043 Update Golang sample 2021-04-16 06:28:33 +09:00
Alec Grieser 7a1133fbf2
Merge pull request #4487 from johscheuer/docker-add-support-multiple-hostnames
Use only one IP address that matches the hostname
2021-04-13 15:12:38 -07:00