Red Hat Enterprise Linux OpenStack
Platform Certification Test Suite 5
User Guide
for use with Certification Test Suite 5
Sankarshan MukhopadhyayJared Morgan
Red Hat Enterprise Linux OpenStack Platform Certification Test Suite 5
User Guide
for use with Certification Test Suite 5
Sankarshan Mukhopadhyay
[e-mail]
Jared Morgan
[e-mail]
Legal Notice
Copyright © 2014 Red Hat.
This document is licensed by Red Hat under the Creative Commons Attribution-
ShareAlike 3.0 Unported License. If you distribute this document, or a modified
version of it, you must provide attribution to Red Hat, Inc. and provide a link to the
original. If the document is modified, all Red Hat trademarks must be removed.
Red Hat, as the licensor of this document, waives the right to enforce, and agrees
not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable
law.
Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix,
Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in
the United States and other countries.
Linux ® is the registered trademark of Linus Torvalds in the United States and
other countries.
Java ® is a registered trademark of Oracle and/or its affiliates.
XFS ® is a trademark of Silicon Graphics International Corp. or its subsidiaries in
the United States and/or other countries.
MySQL ® is a registered trademark of MySQL AB in the United States, the
European Union and other countries.
Node.js ® is an official trademark of Joyent. Red Hat Software Collections is not
formally related to or endorsed by the official Joyent Node.js open source or
commercial project.
The OpenStack ® Word Mark and OpenStack Logo are either registered
trademarks/service marks or trademarks/service marks of the OpenStack
Foundation, in the United States and other countries and are used with the
OpenStack Foundation's permission. We are not affiliated with, endorsed or
sponsored by the OpenStack Foundation, or the OpenStack community.
All other trademarks are the property of their respective owners.
Abstract
The User Guide covers how to use the Red Hat Enterprise Linux OpenStack
Platform Certification Test Suite to submit a third party product for certification.
T able of Cont ent s
Table of Contents
` . . . . . . . . . ct
Ch.ap.t.e.r 1 . In.t.ro.d.u . .i.o.n. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2. . . . . . . . .
`
1.1. Reporting a Product Bug 2
`
1.2. Release Notes 3
` . . . . . . . . . l . o.n. an . . . i . ur . o.n. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4. . . . . . . . .
Ch.ap.t.e.r 2 . In.st.al .at.i . . . d. Co .n.f .g . .at.i .
`
2.1. Partner Prerequisites 4
`
2.2. Install and Configure Red Hat Enterprise Linux OpenStack Platform 5
`
2.3. Install the Test Suite using Subscription Manager 5
`
2.4. Install the Test Suite using RHN or CDN 5
`
2.5. Modify Configuration Files 6
` . . . . . . . . . i i . . o.n. Wo. .kf .l . .
Ch.ap.t.e.r 3. Ce .rt .f .cat.i . . . r . o w. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8. . . . . . . . .
`
3.1. Running Tests 8
`
3.2. Submitting Data Logs 9
`
3.2.1. About Data Logs 9
`
3.2.2. Submitting Data Logs Automatically 9
`
3.2.3. Provide Specific Software Details 10
`
3.2.4. Submitting Data Logs Manually 11
`
3.2.5. Bugzilla Status Reference 12
`
3.3. Responding to Reviewer Feedback (Subset Tests) 12
`
3.3.1. Configuring the Subset Tests 12
`
3.3.2. Running and Uploading Subset Tests 13
` . . . . . . . . p.e.st . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.4. . . . . . . . .
Ch.ap.t.e.r 4 . T .e.m . .
`
4.1. Configuring Tempest with Red Hat Enterprise Linux OpenStack Platform 14
`
4.2. Configuring Tempest for Network, Block Storage, and Object Storage Tests 15
. . . si . n . . . .
Re.vi . .o . .Hist .o.ry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.7. . . . . . . . .
1
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
Chapter 1. Introduction
This guide covers the following topics:
how to download and install the Test Suite
the testing process itself
complementing the Certification Test Suite with results from Continuous Integration
frameworks.
It is intended for use by members of the Red Hat Cloud Infrastructure Partner Network who want
to certify their drivers, plug-ins or software applications with Red Hat Enterprise Linux
OpenStack Platform.
The Test Suite verifies that a third-party product is compatible with Red Hat's OpenStack
technology. It checks component compliance against documented OpenStack APIs for the
specific version of the Red Hat Enterprise Linux OpenStack Platform for which certification is
being pursued. The test suite also determines whether the application packaging and
deployment specifics meet Red Hat specifications.
The current release of the tool collects information about API usage for the Networking, Block
Storage and Object Storage services. Additionally, it logs information about the system
environment including environment variables and run-times. In addition to running the
Certification Test Suite, Red Hat may require additional tests or information to acquire deeper
understanding of the deployment scenario.
Report a bug
1.1. Reporting a Product Bug
Your feedback is crucial to continually improve the Red Hat Certification Program. Open
https://bugzilla.redhat.com/enter_bug.cgi?
product=Red%20Hat%20OpenStack%20Certification% 20Program to create a new bug against
the Red Hat OpenStack Certification Program. When creating your bug report, select from the
following components:
rho sct s
For any defect or enhancement requests against the Red Hat Enterprise Linux
OpenStack Platform Certification Test Suite
Do cument at io n
For any defect or enhancement request against the User Guide for the Certification
Test Suite and the Policy Guide.
cert ificat io n-cat alo g
For any defect or enhancement request against the public listing of Red Hat Enterprise
Linux OpenStack Platform certified components
Report a bug
2
`
Chapt er 1 . Int roduct ion
1.2. Release Notes
The Release Notes available from https://access.redhat.com/site/documentation/en-
US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html/Release_Notes/index.html contain
a list of Known and Resolved Issues relating to the Certification Test Suite.
Report a bug
3
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
Chapter 2. Installation and Configuration
2.1. Partner Prerequisites
Red Hat Enterprise Linux OpenStack Platform must be included as one of the supported
platforms for drivers, plug-ins, or software applications being certified. See the Policy Guide for
more information about certification requirements.
Obt ain an Ent it lement t o Cert ificat io n T est Suit e
Complete this task the first time certification is pursued. Once completed, the bugzilla privileges
and access to software will remain active across Red Hat' OpenStack releases.
Prerequisit es
Partners must meet the following prerequisites to use the Red Hat Enterprise Linux OpenStack
Certification Test Suite.
A corporate account on http://redhat.com.
A bugzilla account (https://bugzilla.redhat.com/)
Membership in the Red Hat Cloud Infrastructure Partner Network.
Understand how to configure the product by reading
https://access.redhat.com/site/documentation/en-
US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html/Installation_and_Configuration_
Guide/index.html
1. Send an email to [e-mail] containing the following
information:
Bugzilla username.
Red Hat Corporate Account username.
The support team will enable the correct entitlement and permissions on the accounts,
and will inform you when this process is complete.
2. Once the support team has fulfilled the request, verify the rho scert _ent er Bugzilla
permission has been enabled on the account. In Bugzilla, click Administration >
Permissions to verify this permission is appended correctly.
3. Once the software entitlement is granted, install Red Hat Enterprise Linux OpenStack
Platform as well as the driver or plug-in to be certified.
For confidentiality reasons, a certification request will only be visible to the account that
created it. Partners are advised to use a shared Bugzilla account to submit and track all
certification activity.
4
`
Chapt er 2 . Inst allat ion and Configurat ion
Report a bug
2.2. Install and Configure Red Hat Enterprise Linux
OpenStack Platform
Before installing and configuring the Certification Test Suite, the corresponding Red Hat
Enterprise Linux Openstack Platform release must be installed.
See https://access.redhat.com/site/documentation/en-
US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html-
single/Installation_and_Configuration_Guide/index.html for instructions relating to installing and
configuring the product for your application environment.
Report a bug
2.3. Install the Test Suite using Subscription Manager
Once the product repositories have been configured, add the corresponding repository for the
Certification Test Suite and install the test suite.
Pro cedure 2.1. Inst all t he T est Suit e using Subscript io n Manager
1. Enable the Test Suite repository.
Fo r Red Hat Ent erprise Linux 7
# subscription-manager repos --enable=rhel-7-server-
openstack-5.0-cts-rpms
Fo r Red Hat Ent erprise Linux 6
# subscription-manager repos --enable=rhel-6-server-
openstack-5.0-cts-rpms
2. Install the package.
# yum install openstack-cts
Report a bug
2.4. Install the Test Suite using RHN or CDN
The following configuration information relates to installing the package from Red Hat Network
(RHN) and Content Delivery Network (CDN) for Red Hat OpenStack 5.0 Certification Test Suite
for Server 7.
See https://access.redhat.com/site/documentation/en-
US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html-
5
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
single/Installation_and_Configuration_Guide/index.html#sect-
Software_Repository_Configuration for information relating to installing OpenStack using RHN
and CDN.
Note
To obtain Red Hat OpenStack 5.0 Certification Test Suite for Server 6, the Channel Label,
Name, and Description information must be modified to point to Server 6.
RHN Channel Info rmat io n
The following parameters are required for RHN channel subscription.
Channel Family
openstack-cts
Channel Label
rhel-x86_64-server-7-ost-5-cts
Channel Name
Red Hat OpenStack 5.0 Certification Test Suite for Server 7
Channel Descript io n
Red Hat OpenStack 5.0 Certification (for Server v7 AMD64/Intel64)
CDN Co nt ent Info rmat io n
The following parameters are required for CDN channel subscription.
Co nt ent Name
Red Hat OpenStack 5.0 Certification Test Suite for Server 7
Co nt ent Label
rhel-7-server-openstack-5.0-cts-rpms
Report a bug
2.5. Modify Configuration Files
Once the Red Hat Enterprise Linux OpenStack Platform Certification Test Suite is downloaded
and installed, the following files must be appropriately modified.
All files contain sections with comment blocks to assist with providing the required values.
Co nfigurat io n Files
api.co nf
Contains subset test configuration flags that allow the full scope of tests to be
disabled as required. This file is usually changed by a partner engineer under direction
from Red Hat Certification Team members.
6
`
Chapt er 2 . Inst allat ion and Configurat ion
bugzilla.co nf
Contains the URL to the Red Hat Bugzilla instance. To automatically report data logs,
the partner engineer must input the login credentials for the Bugzilla account created in
Section 2.1,  Partner Prerequisites .
net wo rk.ini , st o rage.ini , o bject st o rage.ini
These files require the following common information: the username, password, and
tenant name for the administrator and normal users; the IP address of the machine on
which the tests are to be invoked; other information which is commented in the file.
To enable the configuration, uncomment the blocks and provide the information
required.
Report a bug
7
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
Chapter 3. Certification Workflow
The certification workflow operates in the following way:
Run the test suite with options appropriate for the software being certified.
Submit the Data Logs.
Respond to reviewer feedback:
Run further tests if required
Submit new logs.
Report a bug
3.1. Running Tests
The Red Hat Enterprise Linux OpenStack Certification Test Suite is provided as a framework
with the tests for the individual components available by means of options (or a configuration
file).
The current release of the Test Suite is configured to test and certify a plug-in for the Networking,
Block Storage and Object Storage services, and it must be installed and configured on the
hardware on which these services are running. In order to run the Test Suite, the user must have
root user privileges.
See Section 4.2,  Configuring Tempest for Network, Block Storage, and Object Storage Tests
for an alternative method for testing OpenStack services and applications not covered by the
scope of the Test Suite.
Cert ificat io n T est Suit e Co mmands
# rho sct s --sysinfo
Run tests against the system environment, including installed packages, package
repositories, and system services.
# rho sct s --cert ify-net wo rk
Run tests against the Networking plug-in.
# rho sct s --cert ify-st o rage
Run tests against the Block Storage plug-in.
# rho sct s --cert ify-o bject st o rage
Run tests against the Object Storage plug-in.
Data logs generated from the tests form the primary part of certification, and must be submitted
for review. The logs are uploaded and retained in Bugzilla, and are visible to the Red Hat
Certification Team and members of the partner-specific group used to file the certification.
Report a bug
8
`
Chapt er 3. Cert ificat ion Workflow
3.2. Submitting Data Logs
3.2.1. About Data Logs
During the test, the tool collects data for the system infrastructure under test. This non-
exhaustive list of data includes the following:
Hostnames of machines on the systems under test.
IP addresses of machines on the systems under test.
API, and system calls for the services under test.
Configuration files required to set up OpenStack services.
The logs assist the reviewer to develop a deeper understanding of the architecture on which the
software being certified is installed, configured and deployed.
For OpenStack software not covered within the scope of the test suite, logs generated by
Continuous Integration (CI) frameworks such as Tempest and Jenkins can be requested for
review, in addition to logs generated by the Test Suite.
See Also :
Section 4.1,  Configuring Tempest with Red Hat Enterprise Linux OpenStack Platform
Report a bug
3.2.2. Submitting Data Logs Automatically
The Certification Test Suite (CTS) collects test data from the system and stores it in the system
logs while the subset test is running. At the end of each test run, the CTS initiates a reporting
sequence that allows the logs to be automatically uploaded to Bugzilla as a new support
request.
Submit t ing Dat a Lo gs using Aut o mat ed Submissio n
The partner engineer completes this task to automatically submit the required logs to the Red
Hat Certification Team.
Read Section 2.5,  Modify Configuration Files to understand the location of
bugzilla.conf.
Read Section 3.1,  Running Tests to understand how to initiate the required tests.
1. Supply bugzilla partner credentials in bugzilla.conf, using the comments in the file
as a guide.
2. Run the tests required against the software to certify.
9
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
3. When the tests are complete, the CTS will ask for permission to upload logs
automatically and create a review request. Type Y when prompted to instruct CTS to
connect to Red Hat Bugzilla and upload the logs.
The CTS indicates the upload process is successful when a link to the created Bugzilla
is received in the command-line interface.
Report a bug
3.2.3. Provide Specific Software Details
Once a Bugzilla request is created automatically or manually, further information is required to
ensure the Red Hat Certification Team has all the information they need to analyze and process
the Review Request.
Pro vide Det ailed So ft ware Info rmat io n
The partner engineer completes this task to provide the additional information required after the
Bugzilla Testing Request is created initially. Once this information is supplied, the Certification
Testing Team can begin to review and provide comment on the submission.
Prerequisit es
Read Section 2.1,  Partner Prerequisites to understand the partner requirements.
1. Specify the correct component from the following options.
certification-networking-plugin (For certification requests of the Networking
plugins/drivers)
certification-blockstorage-plugin (For certification requests of the Block Storage
plugins/drivers)
certification-objectstorage-plugin (For certification requests of the Object Storage
plugins/drivers)
2. Specify the version number of the Certification Test Suite.
3. Complete the Summary fields with a summary of the bug.
4. Provide the following information in the Description field:
[Partner_Name] certification for [component_name] [component-version] on
[OpenStack_Version].
Link to Product Specification Documentation along with the Marketing name for the
Product.
Link to Installation and Configuration documentation for the product specific to the
Red Hat Enterprise Linux OpenStack Platform.
Link to documentation for orchestration using scripts (for example, Puppet).
Download URL for the software.
10
`
Chapt er 3. Cert ificat ion Workflow
A list of all RPM dependencies (in Name-Version-Release form) required by the
plug-in, and not provided as part of the Red Hat Enterprise Linux OpenStack Platform.
Specific settings for system services to enable complete functionality of the product.
For example, mandatory open ports on the firewall.
Any specific settings for security contexts and profile to enable complete functionality
of the product.
Location of log files.
5. Click Submit to update the bug.
Report a bug
3.2.4. Submitting Data Logs Manually
If certification is being conducted in a non-networked environment (for example, a network-
isolated lab), the Certification Test Suite (CTS) allows logs to be uploaded manually to Bugzilla
when access to a networked PC is available.
Submit t ing Dat a Lo gs using Manual Submissio n
The partner engineer completes this task to manually create a Red Hat Bugzilla testing request,
and submit the required logs to the Red Hat Certification Team.
Prerequisit es
Read Section 3.2.3,  Provide Specific Software Details to understand the additional
information required after initially creating the Bugzilla request.
Read Section 3.1,  Running Tests to understand the CTS commands available.
1. Open https://bugzilla.redhat.com/enter_bug.cgi?
product=Red%20Hat%20OpenStack%20Certification to open the basic bug entry
screen for Red Hat OpenStack Certification.
2. Complete the initial information using Section 3.2.3,  Provide Specific Software Details
as a template for completion, and click Submit to log the Testing Request.
3. Once saved, continue to complete the additional fields as described in Section 3.2.3,
 Provide Specific Software Details and click Submit once more to commit the changes
to the Testing Request.
4. Run the tests required against the software to certify.
5. When the tests are complete, the CTS will ask for permission to upload logs
automatically and create a review request. Type N when prompted to instruct the CTS to
cancel the automated reporting sequence.
The logs are saved as an archive in the /tmp directory of the operating system.
11
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
6. Move the logs to a PC with network access, and upload the generated logs as an
attachment to the Testing Request.
7. Click Submit to save the uploaded logs to the Testing Request.
Report a bug
3.2.5. Bugzilla Status Reference
Once a request is submitted, the Bugzilla status progresses through the following states:
NEW
The Testing Request was submitted successfully, however no reviewer is assigned at
this stage.
ASSIGNED
A reviewer has been assigned to the Testing Request.
MODIFIED
The review is underway. Comments are provided by the Certification Engineer. If extra
information is required from the Partner Engineer, the NEEDINFO flag is used to
request this information.
RELEASE
The review has been completed. The review engineer has signed-off on test results,
and certification listing documentation is being prepared.
CLOSED
The certified software is published to the marketplace.
CLOSED DUPLICAT E
This is a specific status used to indicate a Testing Request already exists for the
partner, and the new submission is either similar in content, or a duplicate.
Report a bug
3.3. Responding to Reviewer Feedback (Subset
Tests)
This release of the Red Hat Enterprise Linux OpenStack Platform Certification Test Suite allows
a specific subset of the tests to be invoked. This feature is useful in cases where the application
has successfully completed all but a few of the tests, and the reviewer requires specific
additional detail for the test. It also has the advantage of reducing the test duration, and the size
of the log files.
Report a bug
3.3.1. Configuring the Subset Tests
12
`
Chapt er 3. Cert ificat ion Workflow
This feature is configured by setting flags in the /etc/rhocts/api.conf file. Setting a flag to
1 will enable the test. To disable a test, set the flag set to 0.
The file contains the [network], [storage] and [objectstorage] sections, each controlling their
respective tests.
The reviewer assigned to the review request will provide instructions relating to which sections
of the file need to be changed, to ensure the test subsets run correctly.
Report a bug
3.3.2. Running and Uploading Subset Tests
Run the subset tests by following the instructions provided by the reviewer. Once the
Certification Test Suite finishes running the subset tests, the data logs will be ready for upload
manually or automatically.
See Also :
Section 3.2.1,  About Data Logs
Report a bug
13
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
Chapter 4. Tempest
4.1. Configuring Tempest with Red Hat Enterprise
Linux OpenStack Platform
To test parts of the system not covered by the Certification Test Suite, partners must install,
configure and use integration test frameworks such as Tempest as part of the certification
process.
Enable T empest wit h PackSt ack
The partner engineer completes this task to configure Tempest. Once configured, additional
tests not covered by the scope of the Certification Test Suite can be run.
All commands in this task must be run as the root user.
Read http://docs.openstack.org/developer/tempest/overview.html to understand Tempest.
Read https://access.redhat.com/site/documentation/en-
US/Red_Hat_Enterprise_Linux_OpenStack_Platform/5/html/Getting_Started_Guide/part-
Deploying_OpenStack_using_PackStack.html to understand how PackStack operates. This
procedure assumes prior knowledge of PackStack.
1. Edit the PackStack generated answer file, and update the tempest configuration:
CONFIG_PROVISION_TEMPEST=y
CONFIG_PROVISION_TEMPEST_REPO_REVISION=master
# packstack --answer-file=[answers-file].txt
2. Configure Tempest by running the following commands:
# cd /var/lib/tempest
# python setup.py install
3. Edit /var/lib/tempest/etc/tempest.conf
Move image_ref and image_ref_alt at the end of file to the [compute] section.
Move public_network_id at the end of file to the [network] section.
4. Edit the following components in the [identity] section:
uri=http://127.0.0.1:5000/v2.0/
uri_v3=http://127.0.0.1:5000/v3.0/
auth_version=v2
14
`
Chapt er 4 . T empest
5. Update the logging section in /var/lib/tempest/etc/tempest.conf.
log_file=[path_to_log_file]
6. Execute the complete set of tempest tests for the current setup
# ./run_tempest.sh -N
Report a bug
4.2. Configuring Tempest for Network, Block
Storage, and Object Storage Tests
Tempest can be configured to run Networking, Block Storage and Object Storage tests. The
following procedures provide more detail about each individual component.
Pro cedure 4.1. Net wo rk (Neut ro n) T est Co nfigurat io n
1. Configure the driver under test in the /etc/neutron/neutron.conf file. Configure
the plug-in according to requirements.
2. Edit the /var/lib/tempest/etc/tempest.conf file.
a. Update backend information in the [network] section.
b. Update the public_router_id object.
c. Update extension tests to be executed in [network-feature-enabled] section.
d. Ensure a comma-separated list is supplied for the discoverable_apis field,
or specify all for default settings.
3. Run # ./run_tempest.sh -N -- run tempest.api.network to start the
networking (Neutron) API tests.
Pro cedure 4.2. Blo ck St o rage (Cinder) T est Co nfigurat io n
1. Configure the volume_driver under test in the /etc/cinder/cinder.conf file.
Note
If testing against multiple backends, follow the steps in the Configure a multiple-
storage back-end section of the OpenStack 5 Installation and Configuration
Guide.
2. Edit the /var/lib/tempest/etc/tempest.conf file.
a. Update backend information in the [volume] section.
15
Red Hat Ent erprise Linux OpenSt ack Plat form Cert ificat ion T est Suit e 5 User Guide
b. Update backend_name, storage_protocol, and vendor_name.
c. Update tests to be executed in the [volume-feature-enabled] section.
3. Run # ./run_tempest.sh -N -- run tempest.api.object_storage to start
the block storage (Cinder) API tests.
Pro cedure 4.3. Object St o rage (swift ) T est Co nfigurat io n
1. Configure swift settings for account, container and object servers.
2. Edit the /var/lib/tempest/etc/tempest.conf file.
a. Update backend information in the [object-storage] section.
b. Update tests to be executed in the [object-storage-feature-enabled] section.
c. Ensure a comma-separated list is supplied for the discoverable_apis field,
or specify all for default settings.
3. Run # ./run_tempest.sh -N -- run tempest.api.object_storage to start
the object storage (Swift) API tests.
Report a bug
16
Revision Hist ory
Revision History
Revisio n 5.0-3 T hu 31 Jul 2014 Jared Mo rgan
BZ#1123109 - Corrected the RHEL7 repository name in Section 2.3,  Install the Test Suite
using Subscription Manager
Revisio n 5.0-2 Mo n 07 Jul 2014 Jared Mo rgan
Prepared for 5.0 GA
17