Features
The following table describes the key features of the EGS platform.
| Main Feature | Sub Feature | Description |
|---|---|---|
| Multi-Cluster and Multi-Cloud Fleet Management | Cluster registration | Supports EKS, GKE, OKE, AKS, Akamai, on-prem cluster registration and management |
| Create or delete slice workspace | Slice workspace has one or more namespaces associated with it. | |
| Users can deploy AI workloads in the Slice Workspace namespaces. | ||
| Slice workspace has a GPU VPC. By default, no GPUs are available for slice workspace. | ||
| A GPR (GPU Provision Request) is required to provision GPUs for a slice workspace. | ||
| Multi-Tenancy: Slice Workspaces | One or more namespaces per slice workspace | One or more namespaces can be associated with a workspace. |
| One or more worker clusters per slice workspace | Slice workspace can span across clusters. One or more clusters can be associated with a slice workspace. | |
| Admin Access to Portal | Admin Profile and RBAC | RBAC for admin access |
| Access to Admin Portal | An individual portal for an admin. | |
| User per slice workspace | User per Slice workspace | |
| User Access to Portal | Access to User Portal | An individual portal for a user to manage GPRs and AI workloads within a specific slice workspace. |
| User profile and RBAC | User Slice RBAC and manage that slice workspace. | |
| Inventory Management | GPU node pools, inventory list, and allocation status | Inventory management. Nvidia GPUs. GPU node pools. By default, all GPU pools/nodes are managed by EGS. |
| GPU Provisioning Requests (GPRs) | Create/delete/update GPR | GPU provisioning request for a slice workspace. Request to include a number of parameters such as GPU type, number, memory, nodes, cluster, and so on. |
| GPR Queue Management | GPR queue/list | List/view GPR queues, and GPR status. |
| Change GPR priority | Change the priority of a queued GPR. | |
| Early release a GPR | Early release of a provisioned GPR. Release GPUs provisioned to a slice workspace. | |
| Multi-Cluster support. GPR scoped to single cluster | Users can select the cluster for GPR. | |
| GPU Allocation Management | Full node allocations | Full node allocated to only one slice workspace |
| Idle timeout of allocations | After Idle timeout, return the GPUs to free pool. | |
| EGS Core API Support | API Tokens | Users can create/manage API tokens. Needed to access Core APIs. |
| Core APIs | EGS core APIs | |
| EGS Core SDK Support | Python SDK package | Supports all the EGS Core APIs |
| Deep AI Workload Observability | AI Workloads visualization | AI workload details in a slice workspace |
| Pods/Model details | Pods/jobs running in the workspace. List of pods/jobs and model details. | |
| GPU usage/allocations | GPU/CPU allocations and usage for the workspace. | |
| Dashboards | GPU Allocations/Usage and GPRs | GPU node/GPUs allocated/available, cost associated with allocations, total cost, leaderboards, different cluster/GPU type/and so on. |
| Inference Endpoints | Deploy and manage Inference Endpoints | Deploy one or more Inference Endpoints in a slice workspace. |
| Supports HF/Mistral/other Models; Model Selection | Support for various commercial/open source LLM models. | |
| Supports custom model deployment | Upload custom model for deployment | |
| GPU provisioning for Inference endpoints | GPU presets or advanced custom configuration | |
| OpenAI Compatible APIs | OpenAI compatible APIs for inference - chat/completions/and so on | |
| Events | GPR Events | Visualization of events |
| Admin Workflows | Register clusters, and manage slice workspaces and users | Using the Admin access token |
| Manage Slice Workspaces, GPRs, and Inference Endpoints | Manage Slice Workspaces, GPRs, and Inference Endpoints. | |
| User Workflows | Manage GPRs and Inference Endpoints | Using the User access token |