Introduction
This page describes the architecture of the Tycoon system at both the design and implementation levels.
Design Principles
The architecture of this system is guided by the following design principles:
- Unbundled management. This suggests that the core system only provide the bare minimum of functionality to ensure security and meet quality-of-service requirements. All other functionality, such as configuration management, distributed monitoring, etc. can be provided by services layered on the core system. This principle comes from PlanetLab.
- End-to-end principle. This recommends that functionality be pushed to the edges of the network and higher in the stack, except when it can be fully implemented at the core and at lower layers, or when the lower layer functionality only acts as a performance enhancement for the higher layer functionality. In this context, the resource allocation system is at a lower layer than an application agent, so this principle recommends pushing functionality out of the resource allocator and into application agents and brokers.
- Incentive-based design. An incentive-based design explicitly acknowledges that users, especially those from different organizations, have disparate goals. In cases where these goals could conflict, a mechanism arbitrates among the users. Examples of mechanisms are the New York Stock Exchange and EBay.
- Distributed Mechanisms. Related to the previous principle of incentive-based design is the idea of distributed mechanisms, where the administration of the mechanism is distributed amongst many entities. An example is a bazaar, where each merchant runs his or her own price-setting mechanism, in contrast to EBay, where all merchants must use EBay's mechanism.
- Self-responsibility. This is also related to incentive-based design. The idea is to assign responsibility for an action to the entity that requires the action. The only way to transfer responsibility to another entity is by transferring the incentive to see an action completed. For example, if a user's program needs to perform a database query, it is his or her responsibility to get it done. The only way to transfer this responsibility is for the user to give currency to a service that performs database queries. In contrast, some architectures assign responsibility for some kinds of actions to an entity other than the one who desires it, without explicit transfer of incentives.
The flip side of self-responsibility is that entities are responsible for the consequences of their actions. If the user's program triggers a network security alarm, he or she is responsible for any consequences.
Components
The architecture is composed of the following components:
- Auctioneer. This component is entirely owned by a single entity and allocates resources owned by that entity. An entity can have multiple auctioneers for scalability. An auctioneer is a combination of an operating system's resource allocators for CPU, memory, disk, etc. and a market mechanism. It autonomously allocates resources for users who pay it. Auctioneers do not communicate with each other.
- Bank. The bank is a database of user accounts. It keeps track of the user's public key and the amount of currency owned by the user. The bank is administered by a trusted third-party. In the extreme, the distributed mechanism principle would suggest that the bank administration be distributed in some way, but distributed currency schemes are complex and outside the scope of this JIP.
- Service Location Service (SLS). The service location service keeps track of the availability and prices of resources and service in the system. The SLS is also administered by a trusted third-party.
- User Agent. User agents act on behalf of one or more users. They query the SLS to find resources and make calculations about which resource are most desirable for the user's application. They could also utilize other services for configuration management, distribution of code and data, etc.
One special form of agent is a broker, which is owned by a third-party and acts on behalf of multiple users. A broker performs the same functions as a single-user agent, but also requires payment to perform its services (see the self-responsibility principle above). Brokers might be useful because they can mitigate price fluctuations by spreading risk among multiple users and over time.
Auctioneer
- /src/Tycoon/Auctioneer.py
- Handles incoming RPCs from a remote AuctioneerClient, makes sure they are secure and well-formed, and processes them into the appropriate format for AuctioneerAllocator. Allows creating and deleting accounts, setting bids, and querying various kinds of status information. Is a subclass of ChildAgent. Calls BankClient, AuctioneerAllocator, !Bid, !KL.Security.Signer, !KL.Security.Keys, and !KL.Utility.Flow.
- /src/Tycoon/AuctioneerAllocator.py
- Allocates resources according to the preferences of the user accounts. Implements all the mechanism design algorithms. Directs a VirtualMachineMonitor to enforce the allocation. Calls Virtualization.VirtualMachineMonitor.py, Virtualization.Resources, ProbabilityDistribution, and WindowStatistics.
- /src/Tycoon/Virtualization/VirtualMachineMonitor.py
- Defines an abstract class for a "virtualization" technology, which could be true virtualization, para-virtualization, UNIX user accounts, or an isolated host.
- /src/Tycoon/Virtualization/Xen.py
- Enforces allocations using the Xen para-virtualization system. Is a subclass of VirtualMachineMonitor. Calls Resources, ResourcesUnique, !OSUtil, XenUtil, Log, TycoonUtility, and !KL.Network.Interface.
- /src/Tycoon/Virtualization/Resources.py
- Defines the different fungible computational resources that are allocated by Tycoon.
- /src/Tycoon/Virtualization/ResourcesUnique.py
- Defines unique resources resources like IP addresses and TCP ports.
- /src/Tycoon/Virtualization/Xen2.py
- Enforces allocations using Xen 2.0. Is a subclass of Xen. Defines differences for Xen 2.0 from the base implementation in Xen.
- /src/Tycoon/Virtualization/Xen3.py
- Enforces allocations using Xen 3.0. Is a subclass of Xen. Defines differences for Xen 3.0 from the base implementation in Xen.
- /src/Tycoon/Virtualization/VServer3.py
- Enforces allocations using the VServer 3.0 system call virtualization system. Not maintained.
- /src/Tycoon/Virtualization/VServer2.py
- Enforces allocations using the VServer 3.0 system call virtualization system. Not maintained.
- /src/Tycoon/Virtualization/PlanetLabNodeManager.py
- Enforces allocations using the PlanetLab NodeManager. Not maintained.
- /src/Tycoon/Virtualization/XenUtil.py
- Contains Xen utility functions. Calls !OSUtil.
- /src/Tycoon/Virtualization/OSUtil.py
- Contains operating system utility functions used to support Xen virtualization. Calls !KL.Utility.Flow.
- /src/Tycoon/Virtualization/Log.py
- Logs messages for Xen.
Bank
TBD
Service Location Service
- /src/Tycoon/ServiceLocationService.py
- Implements the Service Location Service. Is a subclass of ChildAgent.
User Agent
- /src/Tycoon/Tycoon.py
- Implements the Tycoon command line interface. Calls AuctioneerClient, BankClient, !SLSClient, Bid, TycoonAgent, BestResponse, BestResponseAllocator, User, KL.Security.Keys, and KL.Utility.Options.
- /src/Tycoon/TycoonAgent.py
- TBD
- /src/Tycoon/BestResponse.py
- Implements Li Zhang's Best Response algorithm. Given a total budget, a deadline, and optionally maximum number of hosts to bid on, the Best Response algorithm calculates how much money should be bid on the different hosts in the system. By default the spot market price as reported by the SLS is used to determine the bids. An additional cushion of money can be spent to account for future expected price developments either using normal distribution or maximum entropy distribution approximations. Also implements the fundamental client interfaces to prediction: get_price(performance,probability), get_performance(probability,price), get_probability(performance,price).
- /src/Tycoon/BestResponseAllocator.py
- Uses BestResponse to place the resulting bids on the hosts using the AuctioneerClient.
- /src/Tycoon/User.py
- Helps users setup their accounts. Calls KL.Security.Keys.
Utility Code
- /src/Tycoon/AuctioneerClient.py
- Provides an RPC client stub for call a remote Auctioneer's methods. Calls KL.Network.XMLRPCClient and KL.Security.Keys.
- /src/Tycoon/SLSClient.py
- Provides an RPC client stub for calling a remote ServiceLocationService's methods. Calls KL.Network.XMLRPCClient.
- /src/Tycoon/BankClient.py
- Provides an RPC client stub for call a remote Bank's methods. Calls KL.Network.XMLRPCClient.
- /src/Tycoon/Bid.py
- Defines a bid. Provides methods to serialize and unserialize a bid.
- /src/Tycoon/ChildAgent.py
- Used by processes that can be turned in a XMLRPC server daemon (i.e., a process without a controlling terminal that responds to XMLRPC calls). Calls KL.Network.XMLRPCServer, KL.Process.Daemonize, and KL.Process.SIGHUP.
- /src/Tycoon/ProbabilityDistribution.py
- Implements a Histogram or bin-based approximation of a probability distriibution function. The number of bins are fixed, but the ranges of values contained in each bin are set dynamically.
- /src/Tycoon/WindowStatistics.py
- Keeps track of the statistical moments around the origin for some time series data. An approximation is used to keep track of moments in moving time windows (e.g. mean or standard deviation during the last hour or day). Also implements a moving time window of a ProbabilityDistribution.
- /src/Tycoon/Utility.py
- Contains utility functions to create directories, etc.
from the KL library:
- /KL/Network/XMLRPCServer.py
- TBD
- /KL/Security/Keys.py
- TBD
- /KL/Utility/Flow.py
- Provides code to wrap Twisted's Deferreds so that asynchronous functions can be written as a Python generator.
- /KL/Process/Daemonize.py
- TBD
- /KL/Process/SIGHUP.py
- TBD
