4.1. Requirements

Any operation or management request must be authenticated with a signed request via Signature Version 2 or 4 of the Amazon S3 protocol of the corresponding S3 system user. You can create system users on any storage node in the cluster with the ostor-s3-admin create-user -S command and parameter -e specifying the user email address:

# ostor-s3-admin create-user -S -e user@example.com
UserEmail:user@example.com
UserId:a14040e0b2ef8b28
KeyPair[0]:S3AccessKeyId:a14040e0b2ef8b28FZZ8
KeyPair[0]:S3SecretAccessKey:dbwTnQTW602aAAdq8DQVFzB6yrTCFTNiGB8C8RFA
Flags:system

With this user you will authenticate further REST API requests managing the S3 cluster. You can create multiple system accounts for different management operations.

4.1.1. Configuration

In addition, you need to create Virtuozzo Infrastructure Platform directories to modify the default functionality.

Change to the document root directory of your WHMCS server (e.g., /srv/http) and create the following directories in it:

  • whmcs/includes/staas_scripts,
  • whmcs/admin/staas_scripts.

Change to the directory whmcs/includes/staas_scripts.

The first file you need to create includes the S3 configuration. Create a configuration file S3_getConfig.php with the following contents, replacing variables as follows:

  • s3_key with your S3AcessKeyId,
  • s3_secret with your S3SecretAccessKey,
  • s3_gateway with your configured S3 gateway address, and
  • whmcs_username with your WHMCS admin username.
<?php

// Return array with default configuration.
if (!function_exists('S3_getConfig')) {
    function S3_getConfig() {

        // s3 login.
        $vars['s3_key'] = "939e2ac6916b57082P9O";
        $vars['s3_secret'] = "tVYF3kZD9zcTtl6q6QDTHaZKM2nuq4xVcl8ikJpd";

        // s3 gateway.
        $vars['s3_gateway'] = "http://s3.example.com";

        // whmcs login.
        $vars['whmcs_username'] = "admin";

        // Return config array.
        return $vars;
    }
}

?>

4.1.2. Includes

Shared functions required by API operations are provided in a number of standalone PHP include files. The first file returns the client information (e.g., email address) which further S3 API user management requests need for various operations. Create a file S3_getClient.php with the following contentss:

<?php

// API request to get whmcs client information.
if (!function_exists('S3_getClient')) {
    function S3_getClient($userid, $whmcs_username) {

        // Get client details for user email.
        $command = 'GetClientsDetails';
        $data = array(
            'clientid' => $userid,
        );
        $results = localAPI($command, $data, $whmcs_username);

        // Return client information.
     return $results;
    }
}

?>

The next file adds notes to the client in WHMCS with the S3 access key pairs whenever a new user or access key pair is created. Create a file S3_addClientNote.php with the following contentss:

<?php

// API request to add note to client in whmcs.
if (!function_exists('S3_addClientNote')) {
    function S3_addClientNote(
        $userid,
        $whmcs_username,
        $s3_client_userid,
        $s3_client_key,
        $s3_client_secret
    ) {

        // Add note only for non-empty users.
        if (!empty($s3_client_userid)) {

            // Add note with the s3 access key and s3 secret.
            $command = 'AddClientNote';
            $data = array(
                'userid' => $userid,
                'notes' =>
                    "UserId: " . $s3_client_userid . "\n" .
                    "AWSAccessKeyId: " . $s3_client_key . "\n" .
                    "AWSSecretAccessKey: " . $s3_client_secret,
            );
            localAPI($command, $data, $whmcs_username);
        }
    }
}

?>

The next file removes notes from the client in WHMCS with the S3 access key pairs whenever a user or access key pair is removed. Create a file S3_delClientNote.php with the following contents:

<?php

// whmcs database access.
use WHMCS\Database\Capsule;

// API request to remove note from client in whmcs.
if (!function_exists('S3_delClientNote')) {
    function S3_delClientNote(
        $userid,
        $whmcs_username,
        $s3_client_userid,
        $s3_client_key
    ) {

        // Delete notes in database.
        $db = Capsule::connection()->getPdo();
        $db->exec('
            DELETE FROM
              tblnotes
            WHERE
              userid = ' . $userid . '
            AND
              note LIKE "%' . $s3_client_userid . '%"
            AND
              note LIKE "%' . $s3_client_key . '%"'
        );
    }
}

?>

The last file is the cURL library for sending GET, PUT, POST, and DELETE requests. Create a file S3_requestCurl.php with the following contents:

<?php

// API request to s3 gateway.
if (!function_exists('S3_requestCurl')) {
    function S3_requestCurl($s3_key, $s3_secret, $s3_gateway, $s3_query, $method) {

        // Prepare signature.
        $s3_host  = parse_url($s3_gateway, PHP_URL_HOST);
        $s3_date  = date(DATE_RFC2822);

        // Generate signature.
        $s3_signature = hash_hmac('sha1', $method . "\n\n\n" . $s3_date . "\n" .
            current(explode('&', $s3_query)), $s3_secret, true);
        $s3_signature = base64_encode($s3_signature);

        // Curl init.
        $s3_curl = curl_init($s3_gateway . $s3_query);

        // Curl options.
        switch ($method) {
            case "PUT":
                curl_setopt($s3_curl, CURLOPT_PUT, 1);
                break;
            case "POST":
                curl_setopt($s3_curl, CURLOPT_POST, 1);
                break;
            case "DELETE":
                curl_setopt($s3_curl, CURLOPT_CUSTOMREQUEST, "DELETE");
                break;
        }
        curl_setopt($s3_curl, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($s3_curl, CURLOPT_URL, $s3_gateway . $s3_query);
        curl_setopt($s3_curl, CURLOPT_HTTPHEADER, array(
            'Host: ' . $s3_host,
            'Date: ' . $s3_date,
            'Authorization: AWS ' . $s3_key . ':' . $s3_signature,
            'Content-Type:',
            'Expect:',
        ));

        // Call.
        $response = curl_exec($s3_curl);
        $response = json_decode($response, true);

        // Curl deinit.
        curl_close($s3_curl);

        // Return response.
        return $response;
    }
}

?>

4.1.3. Hooks

Hooks allow you to execute custom code when certain events occur in WHMCS. You will need to add S3-related action links to the admin page in WHMCS.

Change to the directory whmcs/includes/hooks and create a file S3_adminAreaClientSummaryActionLinks.php with the following contents:

<?php

// Modify other actions admin page.
function S3_adminAreaClientSummaryActionLinks($vars) {

    // Create additional links.
    $result[] = '<b>S3 - User Management</b>';
    $result[] = '<a href="staas_scripts/S3_createUser.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Create User</a>';
    $result[] = '<a href="staas_scripts/S3_deleteUser.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Delete User</a>';
    $result[] = '<a href="staas_scripts/S3_enableUser.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Enable User</a>';
    $result[] = '<a href="staas_scripts/S3_disableUser.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Disable User</a>';
    $result[] = '<a href="staas_scripts/S3_generateAccessKey.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Generate Access Key</a>';
    $result[] = '<a href="staas_scripts/S3_revokeAccessKey.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Revoke Access Key</a>';
    $result[] = '<a href="staas_scripts/S3_queryUser.php?userid=' .
        $vars['userid'] . '"><img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> Query User (on/off)</a>';
    $result[] = '<a href="staas_scripts/S3_listUsers.php">
        <img src="http://logo.acronis.com/ogimage.png"
            width="16" height="16" border="0" align="absmiddle" /> List Users (on/off)</a>';
    $result[] = '&nbsp;';
    $result[] = '<b>S3 - User Limits Management</b>';
    $result[] = '
        <form>
            <input name="userid" type="hidden" value="' . $vars['userid'] . '">
            <input name="ops-value" size="4">
            <select name="ops-name">
                <option>default</option>
                <option>get</option>
                <option>put</option>
                <option>list</option>
                <option>delete</option>
            </select> ops/s
            <br />
            <input name="bandwidth-value" size="4">
            <select name="bandwidth-name">
                <option>out</option>
            </select> bandwidth/s
            <br />
            <button type="submit"
                formaction="staas_scripts/S3_setLimitsForUser.php">Set</button>
            <button type="submit"
                formaction="staas_scripts/S3_getLimitsForUser.php">Get</button>
            <button type="submit"
                formaction="staas_scripts/S3_deleteLimitsForUser.php">Delete</button>
        </form>
    ';
    $result[] = '&nbsp;';
    $result[] = '<b>S3 - Bucket Limits Management</b>';
    $result[] = '
        <form>
            <input name="userid" type="hidden" value="' . $vars['userid'] . '">
            <input name="ops-value" size="4">
            <select name="ops-name">
                <option>default</option>
                <option>get</option>
                <option>put</option>
                <option>list</option>
                <option>delete</option>
            </select> ops/s
            <br />
            <input name="bandwidth-value" size="4">
            <select name="bandwidth-name">
                <option>out</option>
            </select> bandwidth/s
            <br />
            <input name="bucket" size="4"> bucket name
            <br />
            <button type="submit"
                formaction="staas_scripts/S3_setLimitsForBucket.php">Set</button>
            <button type="submit"
                formaction="staas_scripts/S3_getLimitsForBucket.php">Get</button>
            <button type="submit"
                formaction="staas_scripts/S3_deleteLimitsForBucket.php">Delete</button>
        </form>
    ';
    $result[] = '&nbsp;';
    $result[] = '<b>S3 - Usage Statistics</b>';
    $result[] = '
        <a href="staas_scripts/S3_listStatsObjects.php">
            <img src="http://logo.acronis.com/ogimage.png"
                width="16" height="16" border="0" align="absmiddle" />
                    List Statistics Objects (on/off)
        </a>
        <p>
            <form>
                <input name="object" size="15"> object name
                <br />
                <button type="submit"
                    formaction="staas_scripts/S3_getStatsForObject.php">Get</button>
                <button type="submit"
                    formaction="staas_scripts/S3_deleteStatsForObject.php">Delete</button>
            </form>
        </p>
    ';
    $result[] = '&nbsp;';

    // Return links.
    return $result;
}

// Modify admin area.
add_hook('AdminAreaClientSummaryActionLinks', 1, "S3_adminAreaClientSummaryActionLinks");
?>

The last file extends the admin summary page and displays S3 user information as well as user and bucket limits if the corresponding links are clicked. Create a file S3_adminAreaClientSummaryPage.php with the following contents:

<?php

// Modify admin client summary to show S3 information.
function S3_adminAreaClientSummaryPage($vars) {

    // Sane default.
    $result = '
    <div class="row client-summary-panels">
    ';

    // Show users.
    if ($_SESSION['s3_list_users'] == 1) {

        // Table header.
        $result = $result . '
        <div class="col-lg-6 col-sm-12">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Users List
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>UserId</b></td>
                        <td><b>UserEmail</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_list'] as $s3_row) {
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $s3_row['UserId'] . '</td>
                        <td>' . $s3_row['UserEmail'] . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Show user.
    if ($_SESSION['s3_query_user'] == 1) {

        // Table header.
        $result = $result . '
        <div class="col-lg-6 col-sm-12">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Information for User: ' . $_SESSION['s3_userid'] . '
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>AWSAccessKeyId</b></td>
                        <td><b>AWSSecretAccessKey</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_aws_access_keys'] as $s3_row) {
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $s3_row['AWSAccessKeyId'] . '</td>
                        <td>' . $s3_row['AWSSecretAccessKey'] . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Table footer and next header.
    $result = $result . '
    </div>
    <div class="row client-summary-panels">
    ';

    // Show statistics list.
    if ($_SESSION['s3_stat_objects'] == 1) {

        // Table header.
        $result = $result . '
        <div class="col-lg-6 col-sm-12">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Statistics List
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>Object Name</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_stat']['items'] as $s3_object) {
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $s3_object . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Show limits for user.
    if (!empty($_SESSION['s3_limits_user'])) {

        // Table header.
        $result = $result . '
        <div class="col-lg-3 col-sm-6">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Limits for User
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>Type</b></td>
                        <td><b>Name</b></td>
                        <td><b>Value</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_limits_user'] as $s3_limits => $s3_value) {
            list($s3_type, $s3_limit) = explode(":", $s3_limits);
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $s3_type . '</td>
                        <td>' . $s3_limit . '</td>
                        <td>' . $s3_value . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Show limits for bucket.
    if (!empty($_SESSION['s3_limits_bucket'])) {

        // Table header.
        $result = $result . '
        <div class="col-lg-3 col-sm-6">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Limits for Bucket: ' . $_SESSION['s3_bucket'] . '
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>Type</b></td>
                        <td><b>Name</b></td>
                        <td><b>Value</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_limits_bucket'] as $s3_limits => $s3_value) {
            list($s3_type, $s3_limit) = explode(":", $s3_limits);
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $s3_type . '</td>
                        <td>' . $s3_limit . '</td>
                        <td>' . $s3_value . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Table footer and next header.
    $result = $result . '
    </div>
    <div class="row client-summary-panels">
    ';

    // Show statistics for object.
    if (!empty($_SESSION['s3_object_statistic'])) {

        // Table header.
        $result = $result . '
        <div class="col-lg-12 col-sm-24">
            <div class="clientssummarybox">
                <div class="title">
                    S3 Statistics for Object: ' . $_SESSION['s3_object'] . '
                </div>
                <table class="clientssummarystats" cellspacing="0" cellpadding="2">
                    <tr>
                        <td><b>fmt_version</b></td>
                        <td><b>service_id</b></td>
                        <td><b>start_ts</b></td>
                        <td><b>period</b></td>
                        <td><b>bucket</b></td>
                        <td><b>epoch</b></td>
                        <td><b>user_id</b></td>
                        <td><b>tag</b></td>
                        <td><b>put</b></td>
                        <td><b>get</b></td>
                        <td><b>list</b></td>
                        <td><b>other</b></td>
                        <td><b>uploaded</b></td>
                        <td><b>downloaded</b></td>
                    </tr>
        ';

        // One row per access key pair.
        foreach ($_SESSION['s3_object_statistic']['items'] as $s3_object) {
            $result = $result . '
                    <tr class="altrow">
                        <td>' . $_SESSION['s3_object_statistic']['fmt_version']. '</td>
                        <td>' . $_SESSION['s3_object_statistic']['service_id']. '</td>
                        <td>' . $_SESSION['s3_object_statistic']['start_ts']. '</td>
                        <td>' . $_SESSION['s3_object_statistic']['period']. '</td>
                        <td>' . $s3_object['key']['bucket'] . '</td>
                        <td>' . $s3_object['key']['epoch'] . '</td>
                        <td>' . $s3_object['key']['user'] . '</td>
                        <td>' . $s3_object['key']['tag'] . '</td>
                        <td>' . $s3_object['counters']['ops']['put'] . '</td>
                        <td>' . $s3_object['counters']['ops']['get'] . '</td>
                        <td>' . $s3_object['counters']['ops']['list'] . '</td>
                        <td>' . $s3_object['counters']['ops']['other'] . '</td>
                        <td>' . $s3_object['counters']['net_io']['uploaded'] . '</td>
                        <td>' . $s3_object['counters']['net_io']['downloaded'] . '</td>
                    </tr>
            ';
        }

        // Table footer.
        $result = $result . '
                </table>
            </div>
        </div>
        ';
    }

    // Table footer.
    $result = $result . '
    </div>
    ';

    // Return table.
    return $result;
}

// Modify admin area.
add_hook('AdminAreaClientSummaryPage', 1, "S3_adminAreaClientSummaryPage");
?>

4.1.4. Statistics

You need to have statistics collection enabled on your S3 gateway. The S3 gateway will save the statistics as regular storage objects. On each S3 storage node, create a file /var/lib/ostor/local/gw.conf with the following contents:

# Enable usage statistics collection.
S3_GW_COLLECT_STAT=1

Restart the S3 storage service to apply the configuration changes. Run the following command on all S3 storage nodes:

# systemctl restart ostor-agentd.service

Now you can login to WHMCS. Additional links and S3 management options will be shown in the Client Profile section.

../_images/stor_saas_whmcs_integration2.png