Setting up the benchmark for S3

To set up the benchmark for the S3 resource, install and configure the GOSBench tool.

Prerequisites

To install and configure GOSBench

  1. Download and extract the tool from GitHub on each load generator and the coordinator node:

    # curl -OL https://github.com/mulbc/gosbench/releases/download/v0.4/gosbench_0.4_Linux_x86_64.tar.gz
    # tar -xzf gosbench_0.4_Linux_x86_64.tar.gz
  2. On the coordinator node, create the file gosbench_script.yaml with the following configuration:

    s3_config:
      - access_key: <ACCESS_KEY>
        secret_key: <SECRET_KEY>
        region: eu-central-1
        endpoint: <S3_ENDPOINT_URL>
        skipSSLverify: true
    
    grafana_config:
      endpoint: https://<GRAFANA_ENDPOINT>
      username: admin
      password: <ADMIN_PASSWORD>
    
    tests:
      - name: write test
        read_weight: 0
        existing_read_weight: 0
        write_weight: 100
        delete_weight: 0
        list_weight: 0
        objects:
          size_min: 1
          size_max: 100000
          part_size: 0
          size_distribution: random
          unit: KB
          number_min: 10
          number_max: 10
          number_distribution: constant
        buckets:
          number_min: 1
          number_max: 10
          number_distribution: constant
        bucket_prefix: 1255gosbenchobject_
        prefix: obj
        stop_with_runtime: 900s
        stop_with_ops:
        workers: <NUMBER_OF_WORKERS>
        workers_share_buckets: True
        parallel_clients: 3
        clean_after: True
    
      - name: read test
        read_weight: 100
        existing_read_weight: 0
        write_weight: 0
        delete_weight: 0
        list_weight: 0
        objects:
          size_min: 1
          size_max: 100000
          part_size: 0
          size_distribution: random
          unit: KB
          number_min: 10
          number_max: 10
          number_distribution: constant
        buckets:
          number_min: 1
          number_max: 10
          number_distribution: constant
        bucket_prefix: 1255gosbenchobject_
        prefix: obj
        stop_with_runtime: 900s
        stop_with_ops:
        workers: <NUMBER_OF_WORKERS>
        workers_share_buckets: True
        parallel_clients: 3
        clean_after: True
  3. Update the required fields accordingly, in particular, these fields:

    • access_key is the S3 access key to access the resource.
    • secret_key is the S3 secret key to access the resource,
    • endpoint in the s3_config section is the S3 endpoint URL to access the resource.
    • endpoint in the grafana_config section is the Grafana dashboard URL, which is https://<admin_panel_IP>:8888/grafana/d/dashboard-directory-en-US/.
    • password is the Grafana administrator password.
    • workers is the number of load generators. Note that there are multiple occurrences of this parameter that need to be set to the same value.

For more details, refer to the GOSBench documentation.