28 KiB
Devops client
Devops client is a ruby gem.
Table of contents
Installation
Devops client requirements:
- ruby v1.9.3 or higher
Client can be installed by following command
$ sudo gem install devops-client.gem --no-ri --no-rdoc
After gem installation new command will be available in your system
$ devops
If command wasn't found then necessary to check ruby environment
$ gem environment
And add "EXECUTABLE DIRECTORY" into $PATH
Devops shows help if invoked without parameters:
$ devops
Usage: /usr/bin/devops command [options]
Commands:
Bootsrap templates:
templates list
Deploy:
deploy NODE_NAME [NODE_NAME ...]
Filters:
filter image add ec2|openstack IMAGE [IMAGE ...]
filter image delete ec2|openstack IMAGE [IMAGE ...]
filter image list ec2|openstack
Flavor:
flavor list PROVIDER
Group:
group list PROVIDER
Image:
image create
image delete IMAGE
image list [provider] [ec2|openstack]
image show IMAGE
image update IMAGE FILE
Key:
key add KEY_NAME FILE
key delete KEY_NAME
key list
Network:
network list PROVIDER
Project:
project create PROJECT_ID
project delete PROJECT_ID [DEPLOY_ENV]
project deploy PROJECT_ID [DEPLOY_ENV]
project list
project multi create PROJECT_ID
project servers PROJECT_ID [DEPLOY_ENV]
project set run_list PROJECT_ID DEPLOY_ENV [(recipe[mycookbook::myrecipe])|(role[myrole]) ...]
project show PROJECT_ID
project update PROJECT_ID FILE
project user add PROJECT_ID USER_NAME [USER_NAME ...]
project user delete PROJECT_ID USER_NAME [USER_NAME ...]
Provider:
provider list
Script:
script list
script add SCRIPT_NAME FILE
script delete SCRIPT_NAME
script run SCRIPT_NAME NODE_NAME [NODE_NAME ... ]
script command NODE_NAME 'sh command'
Server:
server add PROJECT_ID DEPLOY_ENV IP SSH_USER KEY_ID
server bootstrap INSTANCE_ID
server create PROJECT_ID DEPLOY_ENV
server delete NODE_NAME [NODE_NAME ...]
server list [chef|ec2|openstack]
server pause NODE_NAME
server show NODE_NAME
server unpause NODE_NAME
Tag:
tag create NODE_NAME TAG_NAME [TAG_NAME ...]
tag delete NODE_NAME TAG_NAME [TAG_NAME ...]
tag list NODE_NAME
User:
user create USER_NAME
user delete USER_NAME
user grant USER_NAME [COMMAND] [PRIVILEGES]
user list
user password USER_NAME
Detailed help for each command can be shown by passing --help to command line.
First run
During first, run devops will detect that its configuration file is absent and will show warning and ask for required parameters: First step is to enter server's host and port:
WARN: File '~/.devops/devops-client.conf' does not exist
Language: ru
Devops service host: <host>:7070
Default API version (v2.0):
Username: my_user
Password: my_password
Configuration file '~/.devops/devops-client.conf' is created
Also necessary to enter API version (current is v2.0) and credentials. After these questions configuration file will be created.
Commands
After running some commands, devops client might show information in JSON format and ask for confirmation. User can approve or decline operation.
Any command has additional options:
| Option | Desciption |
|---|---|
| -h, --help | Show help |
| -c, --config FILE | Specify devops client config file (/home/my_user/.devops/devops-client.conf) |
| -v, --version | devops client version |
| --host HOST | devops service host address (devops-server-host:devops-server-port) |
| --api VER | devops service API version (v2.0) |
| --user USERNAME | use USERNAME for authentication |
| --format FORMAT | Output format: 'table', 'json' (table) |
| --completion | Initialize bash completion script |
Templates
$ devops templates
Usage: /usr/bin/devops command [options]
Commands:
Bootsrap templates:
templates list
devops templates list - command will list available templates for bootstrapping virtual machines by Chef
Deploy
Command performs deployment operation by running Chef client on remote server
$ devops deploy
Usage: /usr/bin/devops command [options]
Commands:
Deploy:
deploy NODE_NAME [NODE_NAME ...]
devops deploy - deploys everything on server
Options:
| Option | Description |
|---|---|
| --tag TAG1,TAG2... | Chef tag names, comma separated list of a tags which will be temporary applied to servers. |
Filters
Filters allows to specify cloud VM images and restrict devops to use only them. It is helpful in case of EC2 which has hungreds of images.
$ devops filter
Usage: /usr/bin/devops command [options]
Commands:
Filters:
filter image add ec2|openstack IMAGE [IMAGE ...]
filter image delete ec2|openstack IMAGE [IMAGE ...]
filter image list ec2|openstack
devops filter image add - adds image id to filters devops filter image delete - removes image id (ids) from filters devops filter image list - shows list of available images
Flavor
$ devops flavor
Usage: /usr/bin/devops command [options]
Commands:
Flavor:
flavor list PROVIDER
devops flavor list - lists available virtual machine configurations
Group
$ devops group
Usage: /usr/bin/devops command [options]
Commands:
Group:
group list PROVIDER
devops group list - displays list of security groups
Image
Command allows managing virtual machine images.
$ devops image
Usage: /usr/bin/devops command [options]
Commands:
Image:
image create
image delete IMAGE
image list [provider] [ec2|openstack]
image show IMAGE
image update IMAGE FILE
devops image create - creates image. Client will ask several questions:
Provider: # select cloud provider (e.g., openstack, ec2)
Choose image: # enter image number from a list
The ssh username: # give ssh username for logging in
Bootstrap template (optional): # select bootstrap template
Options:
| Option | Description |
|---|---|
| --provider PROVIDER | Image provider |
| --image IMAGE_ID | Image identifier |
| --ssh_user USER | SSH user name |
| --bootstrap_template TEMPLATE | Bootstrap template |
| --no_bootstrap_template | Do not specify bootstrap template |
devops delete - delete image by ID
devops image list - list available images
devops image list provider ec2|openstack - list available cloud images (filtered by devops)
devops image list ec2|openstack - list available images
devops image show - show image information
devops image update - update image from provided JSON file
Key
Manage keys (SSH certificates) servers.
Key:
key add KEY_NAME FILE
key delete KEY_NAME
key list
devops key add - adds new key with given name KEY_NAME from file FILE
devops key delete - remove key with name KEY_NAME
devops key list - lists available keys
There is at least one system key which cannot be deleted by user. System keys are registered during devops server configuration and not manageable by user)
Network
$ devops network
Usage: /usr/bin/devops command [options]
Commands:
Network:
network list PROVIDER
devops network list PROVIDER - list available cloud networks for given PROVIDER
Project
Command allows to manage projects
$ devops project
Usage: /usr/bin/devops command [options]
Commands:
Project:
project create PROJECT_ID
project delete PROJECT_ID [DEPLOY_ENV]
project deploy PROJECT_ID [DEPLOY_ENV]
project list
project servers PROJECT_ID [DEPLOY_ENV]
project set run_list PROJECT_ID DEPLOY_ENV [(recipe[mycookbook::myrecipe])|(role[myrole]) ...]
project show PROJECT_ID
project update PROJECT_ID FILE
project user add PROJECT_ID USER_NAME [USER_NAME ...]
project user delete PROJECT_ID USER_NAME [USER_NAME ...]
devops project create - create a new project
Client will ask several questions: Deploy environment identifier: # which environment will be created (dev, test, my_env...) At least one environment required for project. Provider: # Cloud provider (openstack, amazon ec2) Security groups (comma separated), like 1,2,3, or empty for 'default': # List of security groups which will be assigned to new VMs in given environment/ Users, you will be added automatically (comma separated), like 1,2,3, or empty: # list of users Flavor: # server configuration Image: # image for virtual machine Subnets (comma separated), like 1,2,3, or empty: # cloud subnets (openstack or Amazon VPC requires at least one) Run list (comma separated), like recipe[mycookbook::myrecipe], role[myrole]: role[test_dev], # roles and cookbooks which will be assigned to virtual machines Enter expires time if necessary (5m, 3h, 2d, 1w, etc): # virtual machine life time (by default forever)
If project already exists then new environment will be added to it
Options:
| Option | Description |
|---|---|
| --groups GROUP_1,GROUP_2... | Security groups (comma separated list) |
| --deploy_env DEPLOY_ID | Deploy enviroment identifier |
| --subnets SUBNET,SUBNET... | Subnets identifier for deploy enviroment (ec2 - only one sybnet, openstack - comma separated list) |
| --flavor FLAVOR | Specify flavor for the project |
| --image IMAGE_ID | Specify image identifier for the project |
| --run_list RUN_LIST | Run list (comma separated), like recipe[mycookbook::myrecipe], role[myrole]: |
| --users USER,USER... | Users for deploy environment control |
| --provider PROVIDER | Provider identifier 'ec2' or 'openstack' |
| --no_expires | Without expires time |
| --expires EXPIRES | Expires time (5m, 3h, 2d, 1w, etc) |
devops project delete - removes project or its environment
devops project deploy - deploys to all servers in a project or in given environment
Options:
| Option | Desciption |
|---|---|
| --servers SERVERS | Servers list (comma separated) |
devops project list - list all available projects
devops project servers - list all running servers in a project
devops project set run_list - update run-list for a project's environment
devops project show - display project info
devops project update - update project from JSON file
devops project user delete - add user to project
Options:
| Option | Description |
|---|---|
| --deploy_env ENV | Add user to deploy enviroment |
devops project user delete - remove user(s) from a project
Options:
| Option | Description |
|---|---|
| --deploy_env ENV | Add user to deploy enviroment |
Provider
$ devops provider
Usage: /usr/bin/devops command [options]
Commands:
Provider:
provider list
devops provider list - Lists available cloud providers registered on devops server
Script
Manages shell scrips for running on servers
$ devops script
Usage: /usr/bin/devops command [options]
Commands:
Script:
script list
script add SCRIPT_NAME FILE
script delete SCRIPT_NAME
script run SCRIPT_NAME NODE_NAME [NODE_NAME ...]
script command NODE_NAME 'sh command'
devops script list - lists available scripts
devops script add - adds new script with name SCRIPT_NAME from file FILE
devops script delete - removes script SCRIPT_NAME
devops script run - runs script with name SCRIPT_NAME on server with node name (on Chef server) NODE_NAME
Options:
| Option | Description |
|---|---|
| --params PARAMS | Comma separated scipt parameters |
devops script command - run shell command on remote server (bash interpreter is used)
Server
$ devops server
Usage: /usr/bin/devops command [options]
Commands:
Server:
server add PROJECT_ID DEPLOY_ENV IP SSH_USER KEY_ID
server bootstrap INSTANCE_ID
server create PROJECT_ID DEPLOY_ENV
server delete NODE_NAME [NODE_NAME ...]
server list [chef|ec2|openstack]
server pause NODE_NAME
server show NODE_NAME
server unpause NODE_NAME
devops server add - adds new server (bare metal, existing,...) to a project with name PROJECT_ID
devops server bootstrap - bootstraps chef on server and runs Chef client with project run list
Options:
| Option | Description |
|---|---|
| -N, --name NAME | Set chef name |
| --bootstrap_template [TEMPLATE] | Bootstrap template (optional) |
devops server create - launches new server in a cloud with project PROJECT_ID and environment DEPLOY_ENV
Options:
| Option | Description |
|---|---|
| -N, --name NAME | Set chef name |
devops server delete - terminates server
Options:
| Option | Description |
|---|---|
| --instance | Delete node by instance id |
devops server list - list servers
devops server pause - put server on pause (only if cloud provider supports it)
devops server show - show detailed information
server unpause - unpause server
Tag
Manages tags on Chef servers. This functionality can be used for changing deploy behavior according to given tags.
$ devops tag
Usage: /usr/bin/devops command [options]
Commands:
Tag:
tag create NODE_NAME TAG_NAME [TAG_NAME ...]
tag delete NODE_NAME TAG_NAME [TAG_NAME ...]
tag list NODE_NAME
devops tag create - create new tag on chef node with name NODE_NAME
devops tag delete - removes tag from chef node with name NODE_NAME
devops tag list - lists all tags on a chef node with name NODE_NAME
User
User management
$ devops user
Usage: /usr/bin/devops command [options]
Commands:
User:
user create USER_NAME
user delete USER_NAME
user grant USER_NAME [COMMAND] [PRIVILEGES]
user list
user password USER_NAME
devops user create - create user with name USER_NAME
Options:
| Option | Description |
|---|---|
| --password PASSWORD | New user password |
devops user delete - remove user with name USER_NAME
devops user grant - grants permissions for user
Available subcommands:
- all
- flavor
- group
- image
- project
- server
- key
- user
- filter
- network
- provider
- script
Available privileges:
- r
- w
- rw
If privileges are not specified then user is not allowed to run command.
If command and privileges are not specified then user's permissions are set to default values.
devops user list - list all users
devops user password - change user's password
Mini HOWTO
Mostly used scenarios described below.
User management
After clean install root user has empty password, lets set it:
$ devops user password root -u root
Enter password for 'root':
Updated
Let's create user test and grant some permissions for working with filters, images, projects and servers:
If system doesn't have users then let's use root user:
$ devops user create test -u root
Password for root:
Enter password for 'test':
Created
By default user has read permissions for filter, image, project, and server operations. Lets give him write permissions:
$ devops user grant test filter rw -u root
Password for root:
Updated
$ devops user grant test image rw -u root
Password for root:
Updated
$ devops user grant test project rw -u root
Password for root:
Updated
$ devops user grant test server rw -u root
Password for root:
Updated
$ devops user grant test user r -u root
Password for root:
Updated
Image management
First step is to add required images to filter. For OpenStack it is OpenStack image id, for EC2 it is AMI.
devops filter image add openstack 78665e7b-5123-4fa8-b39b-d7643ecd8ed7
Next step is to create image and specify required metadata:
$ devops image create
+--------+-----------+
| API version: v2.0 |
| Provider |
+--------+-----------+
| Number | Provider |
+--------+-----------+
| 1 | ec2 |
| 2 | openstack |
+--------+-----------+
Provider: 2
+--------+---------------------------+--------------------------------------+--------+
| API version: v2.0 |
| Images |
+--------+---------------------------+--------------------------------------+--------+
| Number | Name | ID | Status |
+--------+---------------------------+--------------------------------------+--------+
| 1 | centos-6.4-amd64-20130707 | 78665e7b-5123-4fa8-b39b-d7643ecd8ed7 | ACTIVE |
+--------+---------------------------+--------------------------------------+--------+
Image: 1
The ssh username: root
Bootstrap template (optional):
{
"provider": "openstack",
"name": "centos-6.4-amd64-20130707",
"id": "78665e7b-5123-4fa8-b39b-d7643ecd8ed7",
"remote_user": "root"
}
Create image? (y/n):
Project management
Let's create new project 'my_project' with environment 'test'
$ devops project create my_project
Deploy environment identifier: test
+--------+-----------+
| API version: v2.0 |
| Provider |
+--------+-----------+
| Number | Provider |
+--------+-----------+
| 1 | ec2 |
| 2 | openstack |
+--------+-----------+
Provider: 2
System will show security groups. We are selecting what is needed:
+--------+-------------------------------------+----------+------+-------+-----------+-----------------------------+
| API version: v2.0 |
| Groups |
+--------+-------------------------------------+----------+------+-------+-----------+-----------------------------+
| Number | Name | Protocol | From | To | CIDR | Description |
+--------+-------------------------------------+----------+------+-------+-----------+-----------------------------+
| 1 | default | udp | 1 | 65535 | 0.0.0.0/0 | default |
| | | tcp | 1 | 65535 | 0.0.0.0/0 | |
| | | icmp | -1 | -1 | 0.0.0.0/0 | |
+--------+-------------------------------------+----------+------+-------+-----------+-----------------------------+
| 2 | webports | tcp | 8080 | 8080 | 0.0.0.0/0 | web ports |
| | | tcp | 80 | 80 | 0.0.0.0/0 | |
| | | tcp | 8089 | 8089 | 0.0.0.0/0 | |
| | | tcp | 8443 | 8443 | 0.0.0.0/0 | |
| | | tcp | 443 | 443 | 0.0.0.0/0 | |
+--------+-------------------------------------+----------+------+-------+-----------+-----------------------------+
Security groups (comma separated), like 1,2,3, or empty for 'default':
Next step is to users which can work with a project:
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
| API version: v2.0 |
| Users |
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
| | | Privileges |
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
| Number | User ID | Image | Key | Project | Server | User | Script | Filter | Flavor | Group | Network | Provider |
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
| 1 | test | rw | r | rw | rw | r | r | rw | r | r | r | r |
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
| 2 | root | rw | rw | rw | rw | rw | rw | rw | rw | rw | rw | rw |
+--------+------------------+-------+-----+---------+--------+------+--------+--------+--------+-------+---------+----------+
Users, you will be added automatically (comma separated), like 1,2,3, or empty:
Flavor for environment:
+--------+-----------+--------------+------+-------+
| API version: v2.0 |
| Flavors |
+--------+-----------+--------------+------+-------+
| Number | ID | Virtual CPUs | Disk | RAM |
+--------+-----------+--------------+------+-------+
| 1 | c1.large | 8 | 50 | 8192 |
| 2 | c1.medium | 2 | 50 | 2048 |
| 3 | c1.small | 2 | 20 | 1024 |
| 4 | c2.long | 2 | 120 | 4096 |
| 5 | m1.large | 4 | 80 | 8192 |
| 6 | m1.medium | 2 | 40 | 4096 |
| 7 | m1.small | 1 | 20 | 2048 |
| 8 | m1.tiny | 1 | 3 | 512 |
| 9 | m1.xlarge | 8 | 160 | 16384 |
| 10 | m2.long | 2 | 60 | 2048 |
| 11 | snapshot | 2 | 42 | 2048 |
+--------+-----------+--------------+------+-------+
Flavor: 7
Image for virtual machines:
+--------+--------------------------------------+---------------------------+--------------------+-------------+-----------+
| API version: v2.0 |
| Images |
+--------+--------------------------------------+---------------------------+--------------------+-------------+-----------+
| Number | ID | Name | Bootstrap template | Remote user | Provider |
+--------+--------------------------------------+---------------------------+--------------------+-------------+-----------+
| 1 | 78665e7b-5123-4fa8-b39b-d7643ecd8ed7 | centos-6.4-amd64-20130707 | | root | openstack |
+--------+--------------------------------------+---------------------------+--------------------+-------------+-----------+
Image: 1
Network for a virtual machine:
+--------+--------------+-----------------+
| API version: v2.0 |
| Subnets |
+--------+--------------+-----------------+
| Number | Name | CIDR |
+--------+--------------+-----------------+
| 1 | 172.16.223.0 | 172.16.223.0/24 |
| 2 | 172.16.227.0 | 172.16.227.0/24 |
| 3 | LocalNetwork | 172.16.37.0/24 |
| 4 | LocalNetwork | 10.1.98.0/24 |
| 5 | private | 10.0.0.0/24 |
+--------+--------------+-----------------+
Subnets (comma separated), like 1,2,3, or empty: 5
Chef roles for project and environment. By default will be created new role with name PROJECT-ENV and added to runlist. Additional roles and recipes can be specified here.
Run list (comma separated), like recipe[mycookbook::myrecipe], role[myrole]: role[my_project_test],
Just press enter if server lifetime should be infinite.
Enter expires time if necessary (5m, 3h, 2d, 1w, etc):
Assume that we do not need second environment. Just press 'n' here.
Add deploy environment? (y/n): n
{
"deploy_envs": [
{
"identifier": "test",
"provider": "openstack",
"groups": [
"default"
],
"users": [
"test"
],
"flavor": "m1.small",
"image": "78665e7b-5123-4fa8-b39b-d7643ecd8ed7",
"subnets": [
"private"
],
"run_list": [
"role[my_project_test]"
],
"expires": null
}
],
"name": "my_project"
}
Create project? (y/n):
Last question allows reviewing details and confirming for project creation.
Starting new instance
After that we can create servers and apply chef roles:
devops server create my_project test -N my_server_1
'-N' parameter allows to specify chef node name. By default node name will be generated automatically.