Understanding System Architecture

Metacloud Compute contains several main components.

  • The cloud controller—represents the global state and interacts with the other components. The API server acts as the web services front end for the cloud controller. The compute controller provides compute server resources and usually also contains the Compute service.

  • An auth manager—provides authentication and authorization services when used with the Compute system; you can also use Metacloud Identity as a separate authentication service instead.

  • A volume controller—provides fast and permanent block-level storage for the compute servers.

  • The network controller—provides virtual networks to enable compute servers to interact with each other and with the public network. You can also use Metacloud Networking instead.

  • The scheduler—selects the most suitable compute controller to host an instance.

Compute uses a messaging-based, shared nothing architecture. All major components exist on multiple servers, including the compute, volume, and network controllers, and the Object Storage or Image service. The state of the entire system is stored in a database. The cloud controller communicates with the internal object store using HTTP, but it communicates with the scheduler, network controller, and volume controller using Advanced Message Queuing Protocol (AMQP). To avoid blocking a component while waiting for a response, Compute uses asynchronous calls, with a callback that is triggered when a response is received.

Metacloud Architecture

Hypervisors

Compute controls hypervisors through an API server. Selecting the best hypervisor to use can be difficult, and you must take budget, resource constraints, supported features, and required technical specifications into account. However, the majority of Metacloud development is done on systems using KVM hypervisors.

Working with Projects, Users, and Roles

The Compute system is designed to be used by different consumers in the form of projects on a shared system, and role-based access assignments. Roles control the actions that a user is allowed to perform.

Projects are isolated resource containers that form the principal organizational structure within the Compute service. They consist of an individual VLAN, and volumes, instances, images, keys, and users. A user can specify the project by appending project_id to their access key. If no tenant is specified in the API request, Compute attempts to use a project with the same ID as the user.

For projects, you can use quota controls to limit the following:

  • Number of volumes that can be launched.
  • Number of processor cores and the amount of RAM that can be allocated.

  • Floating IP addresses assigned to any instance when it launches. This allows instances to have the same publicly accessible IP addresses.

  • Fixed IP addresses assigned to the same instance when it launches. This allows instances to have the same publicly or privately accessible IP addresses.

Roles control the actions a user is allowed to perform. There are two roles in Metacloud, Admin and Member. When listing roles with the CLI, you see _member_.

A project limits users’ access to particular images. Each user is assigned a user name and password. Keypairs granting access to an instance are enabled for each user, but quotas are set, so that each project can control resource consumption across available hardware resources.

Note
Metacloud uses the term project instead of tenant.

Using Block Storage

Metacloud provides two classes of block storage: ephemeral storage and persistent volume. The decision to use ephemeral or persistent storage is made before Metacloud deployment. In most cases ephemeral storage is used.

Ephemeral Storage

Ephemeral storage includes a root ephemeral volume and an additional ephemeral volume.

The root disk is associated with an instance, and exists only for the life of this very instance. Generally, it is used to store an instance’s root file system, persists across the guest operating system reboots, and is removed on an instance deletion. The amount of the root ephemeral volume is defined by the flavor of an instance.

In addition to the ephemeral root volume, all default types of flavors, except m1.tiny, which is the smallest one, provide an additional ephemeral block device sized between 20 and 160 GB (a configurable value to suit an environment). It is represented as a raw block device with no partition table or file system. A cloud-aware operating system can discover, format, and mount such a storage device. Metacloud Compute defines the default file system for different operating systems as Ext4 for Linux distributions, VFAT for non-Linux and non-Windows operating systems, and NTFS for Windows. However, it is possible to specify any other filesystem type by using virt_mkfs or default_ephemeral_format configuration options.

For example, the cloud-init package included in the Ubuntu stock cloud image, by default, formats this space as an Ext4 file system and mounts it on an /mnt. This is a cloud-init feature and is not a Metacloud mechanism. Metacloud only provisions the raw storage.

Persistent Volume

A persistent volume is represented by a persistent virtualized block device independent of any particular instance, and provided by Metacloud Block Storage.

Only a single configured instance can access a persistent volume. Multiple instances cannot access a persistent volume. This type of configuration requires a traditional network file system to allow multiple instances accessing the persistent volume. It also requires a traditional network file system like NFS, CIFS, or a cluster file system such as GlusterFS. These systems can be built within a Metacloud cluster, or provisioned outside of it, but Metacloud software does not provide these features.

You can configure a persistent volume as bootable and use it to provide a persistent virtual instance similar to the traditional non-cloud-based virtualization system. It is still possible for the resulting instance to keep ephemeral storage, depending on the flavor selected. In this case, the root file system can be on the persistent volume, and its state is maintained, even if the instance is shut down. For more information about this type of configuration, see the OpenStack Configuration Reference.

Note
A persistent volume does not provide concurrent access from multiple instances. That type of configuration requires a traditional network file system like NFS, or CIFS, or a cluster file system such as GlusterFS. These systems can be built within a Metacloud cluster, or provisioned outside of it, but Metacloud does not provide these features.

Using the EC2 Compatible API

In addition to the native compute API, Metacloud provides an EC2-compatible API. This API allows EC2 legacy workflows built for EC2 to work with Metacloud.

Warning
Nova in tree EC2-compatible API is deprecated. The ec2-api project is working to implement the EC2 API.

You can use numerous third-party tools and language-specific SDKs to interact with Metacloud clouds. You can use both native and compatibile APIs. Some of the more popular third-party tools are:

  • Euca2ools—A popular open source command-line tool for interacting with the EC2 API. This is convenient for multi-cloud environments where EC2 is the common API, or for transitioning from EC2-based clouds to Metacloud. For more information, see the Eucalyptus Documentation.
  • Hybridfox—A Firefox browser add-on that provides a graphical interface to many popular public and private cloud technologies, including Metacloud. For more information, see the hybridfox site.
  • boto—Python library for interacting with Amazon Web Services. You can use this library to access Metacloud through the EC2 compatibile API. For more information, see the boto project page on GitHub.
  • fog—A Ruby cloud services library. It provides methods to interact with a large number of cloud and virtualization platforms, including Metacloud. For more information, see the fog site.
  • php-opencloud—A PHP SDK designed to work with most Metacloud-based cloud deployments, as well as Rackspace public cloud. For more information, see the php-opencloud site.

Using the Building Blocks

In Metacloud the base operating system is usually copied from an image stored in the Metacloud Image service. This is the most common case and results in an ephemeral instance that starts from a known template state and loses all accumulated states on virtual machine deletion. It is also possible to put an operating system on a persistent volume in the Metacloud Block Storage volume system. This gives a more traditional persistent system that accumulates states which are preserved on the Metacloud Block Storage volume across the deletion and re-creation of the virtual machine. To get a list of available images on your system, run:

$ openstack image list
+------------+---------------------------------------+--------+
| ID         | Name                                  | Status |
+------------+---------------------------------------+--------+
| <image_id> | cirros-0.3.4-x86_64                   | active |
| <image_id> | ubuntu16.04                           | active |
| <image_id> | Ops Manager 1.7.7                     | active |
+------------+---------------------------------------+--------+

The displayed image attributes are:

  • ID—Automatically generated UUID of the image
  • Name—Free form, human-readable name for image
  • Active—are available for use.
  • Server—For images that are created as snapshots of running instances, this is the UUID of the instance the snapshot derives from. For uploaded images, this field is blank.

Virtual hardware templates are called flavors. The default installation provides five flavors that are configurable by administrators.

For a list of flavors that are available on your system:

$ openstack flavor list
+-------------+-------------------+-------+------+-----------+-------+-----------+
| ID          | Name              |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+-------------+-------------------+-------+------+-----------+-------+-----------+
| <flavor_id> | m1.medium         |  4096 |   40 |         0 |     2 | True      |
| <flavor_id> | m1.xlarge-20      |  8000 |    0 |        30 |     2 | True      |
| <flavor_id> | m1.xlarge         | 16384 |  160 |         0 |     8 | True      |
| <flavor_id> | m1.large          |  8192 |   80 |         0 |     4 | True      |
| <flavor_id> | m1.small          |  2048 |   20 |         0 |     1 | True      |
+-------------+-------------------+-------+------+-----------+-------+-----------+

Compute Service Architecture

These basic categories describe the service architecture and information about the cloud controller.

API Server

At the heart of the cloud framework is an API server, which makes command and control of the hypervisor, storage, and networking programmatically available to users.

The API endpoints are basic HTTP web services which handle authentication, authorization, and basic command and control functions using various API interfaces under the Amazon, Rackspace, and related models. This enables API compatibility with multiple existing tool sets created for interaction with offerings from other vendors. This broad compatibility prevents vendor lock-in.

Message Queue

A messaging queue brokers the interaction between compute nodes (processing), the networking controllers (software which controls network infrastructure), API endpoints, the scheduler (determines which physical hardware to allocate to a virtual resource), and similar components. Communication to and from the cloud controller is handled by HTTP requests through multiple API endpoints.

A typical message passing event begins with the API server receiving a request from a user. The API server authenticates the user and ensures that they are permitted to issue the subject command. The availability of objects implicated in the request is evaluated and, if available, the request is routed to the queuing engine for the relevant workers. Workers continually listen to the queue based on their role, and occasionally their type host name. When an applicable work request arrives on the queue, the worker takes assignment of the task and begins executing it. On completion, a response is dispatched to the queue which is received by the API server and relayed to the originating user. Database entries are queried, added, or removed as necessary during the process.

Compute Worker

Compute workers manage computing instances on host machines. The API dispatches commands to compute workers to complete these tasks:

  • Run instances
  • Delete instances (Terminate instances)
  • Reboot instances
  • Attach volumes
  • Detach volumes
  • Get console output

Network Controller

The Network Controller manages the networking resources on host machines. The API server dispatches commands through the message queue, which are subsequently processed by Network Controllers. Specific operations include:

  • Allocating fixed IP addresses
  • Configuring VLANs for projects
  • Configuring networks for compute nodes