Manage Resource Quotas

Resource quotas enable the cluster admins to enforce limits for CPU, memory, ephemeral storage and number of application pods per namespace on a slice. It requires setting and monitoring the threshold limit and requests of the resources at the slice level.

The cluster admin can manage the usage of compute resources on a slice ensuring that the namespaces get a fair share of the resources. This prevents some namespaces from overusing the resources leaving little or no resources for other namespaces on the same slice.

Requests and limits are the ways Kubernetes uses to manage resources such as CPU and memory. Requests are what the container is assured to get. Limits ensure that a container is restricted to a particular value. You can also set quotas for requests and limits to manage the local ephemeral storage.

A default limit and request can be configured per container and their purpose is described below:

• Default limit per container: This is used for a container in the namespace that does not have resource limits configured. The default limit per container is set at the namespace level.
• Default request per container: This is used for a container in the namespace that does not have resource requests configured. A default request per container can be set at the slice quota that applies to all namespaces on the slice. This default request per container set at the slice level is overridden when it is set at the namespace level.

Enforce Resource Quota

The resource quota is enforced only at the namespace level but you can set quotas for the limits and requests for a slice. The CPU, memory, pod count, and the local ephemeral storage of all namespaces in a slice must be less than or equal to their corresponding limits and requests set for a slice. The admin can check for quota breaches by tracking the violation in usage metrics through PromQL queries and KubeSlice Manager.

While setting the limits and requests for resources, you should consider the application requirements and define the limit and request values. For more information, see quotas.

Resource Quota Configuration Parameters

The following tables describe the configuration parameters used to set quotas in the sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
apiVersion	String	The KubeSlice Controller API version. A set of resources that are exposed together, along with the version. The value must be controller.kubeslice.io/v1alpha1.	Mandatory
kind	String	The name of a particular object schema. The value must be SliceResourceQuotaConfig.	Mandatory
metadata	Object	The metadata describes parameters (names and types) and attributes that have been applied.	Mandatory
spec	Object	The specification of the desired state of an object.	Mandatory

Resource Quota Metadata Parameters

These parameters are related to setting quotas, which are configured in the sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
name	String	The name of the slice resource quota.	Mandatory

Resource Quota Spec Parameters

These parameters are related to the setting quotas, which are configured in the sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
sliceQuota	Object	It contains the resources for which the total limits that can be consumed by the namespaces on this slice. The total limits are configured at a slice level.	Optional
clusterQuota	Object	It contains the name of the worker clusters, and the quotas set for the namespaces on a slice.	Optional

sliceQuota Parameters

These parameters are related to setting the total number of quotas for a slice, which are configured in the sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
resources	Object	The resources contains the limit and request objects that sets limits for CPU, memory, ephemeral storage and pod count (only apply to limits) for all namespaces on a slice.	Optional

resources Parameters

Parameter	Parameter Type	Description	Required
limit	Object	It is the quota object for the limits set for CPU, memory, ephemeral storage, and pod count for all namespaces on a slice.	Optional
request	Object	It is the quota object for the total requests set for CPU, memory, and ephemeral storage for all namespaces on a slice.	Optional
defaultRequestPerContainer	Object	It is the quota object for the default requests per container set for CPU, memory, and ephemeral storage for all namespaces on a slice. For a resource type, the default request per container must be less than or equal to the corresponding request.	Optional

limit Parameters

Parameter	Parameter Type	Description	Required
cpu	String	It sets the CPU limit for all namespaces on a slice. Its value is in millicores and it does not accept negative values.	Optional
memory	String	It sets the total memory limit for all namespaces on a slice. Its value is in Mi and it does not accept negative values.	Optional
podCount	String	It sets the limit for the total number of application pods on a slice.	Optional
ephemeralStorage	String	It sets the total limit for the local ephemeral storage for all namespaces on a slice. Its value is in Mi and it does not accept negative values.	Optional

request Parameters

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the CPU request total for all the namespaces on a slice. Its value is in millicores and it does not accept negative values. The value must be less than or equal to the CPU limit.	Optional
memory	String	It sets the memory request total for all the namespaces on a slice. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the memory limit.	Optional
ephemeralStorage	String	It sets the request for the total ephemeral storage for all the namespaces on a slice. The value must be less than or equal to the ephemeral storage limit.	Optional

defaultRequestPerContainer Parameters

info

This default request per container value holds good to all namespaces. To override this value, you can set the same parameter under namespaceQuota.

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the default CPU request per container for all the pods of a namespace. Its value is in millicores and it does not accept negative values. The value must be less than or equal to the CPU request.	Optional
memory	String	It sets the default memory request per container for all the pods of a namespace. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the memory request.	Optional
ephemeralStorage	String	It sets the default request for the local ephemeral storage per container for all the pods of a namespace. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the ephemeral storage request.	Optional

clusterQuota Parameters

These parameters are related to enforcing the quotas on the namespaces on all the worker clusters connected to a slice. They are configured in the
sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
clusterName	String	It is the name of the worker cluster that contains the namespaces onboarded onto the slice.	Mandatory
namespaceQuota	Object	It contains the configuration of the resources' limits set for all the application pods in the namespace.	Optional

namespaceQuota Parameters

These parameters are related to the quotas enforced for a namespace, which are configured in the sliceResourceQuotaConfig YAML file.

Parameter	Parameter Type	Description	Required
enforceQuota	Boolean	When enabled, it enforces the quota limit at the namespace level. The value should be set to true.	Mandatory
namespace	String	The namespace on which you want to enforce the quota.	Mandatory
resources	Object	The resources contains the limit object that sets limits for CPU, memory, ephemeral storage and pod count (only apply to limits).	Optional

resources Parameters under namespaceQuota

Parameter	Parameter Type	Description	Required
limit	Object	It is the quota object for limits set for CPU, memory, ephemeral storage, and pods count.	Optional
request	Object	It is the quota object for requests set for CPU, memory, and ephemeral storage.	Optional
defaultLimitPerContainer	Object	It is the quota object for default limits per container set for CPU, memory, and ephemeral storage of a namespace.	Optional
defaultRequestPerContainer	Object	It is the quota object for the default requests per container set for CPU, memory, and ephemeral storage of a namespace.	Optional.

limit Parameters under nameSpaceQuota

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the CPU limit for all the pods of a namespace. Its value is in millicores and it does not accept negative values. The value must be less than less than or equal to the CPU limit set under sliceQuota. If you want to set the CPU limit for more than one namespace, then ensure that the total CPU limit of all the namespaces does not exceed the corresponding CPU limit set under sliceQuota.	Optional
memory	String	It sets the memory limit for all the pods of a namespace. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the memory limit set under sliceQuota. If you want to set the memory limit for more than one namespace, then ensure that the total memory limit of all the namespaces does not exceed the corresponding memory limit set under sliceQuota.	Optional
podCount	String	It sets the limit for the total number of application pods that can exist in a namespace. The value must be less than the pod count limit set under sliceQuota. If you want to set the pod count limit for more than one namespace, then ensure that the total pod count limit of all the namespaces does not exceed the corresponding pod count limit set under sliceQuota.	Optional
ephemeralStorage	String	It sets the limit for the local ephemeral storage for all the pods of a namespace. Its value is Mi and it does not accept negative values. The value must be less than or equal to the ephemeral storage limit set under sliceQuota. If you want to set the ephemeral storage limit for more than one namespace, then ensure that the total ephemeral storage limit of all the namespaces does not exceed to the corresponding ephemeral storage limit set under sliceQuota.	Optional

request Parameters under namespaceQuota

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the CPU request total for all the pods of the namespace. Its value is in millicores and it does not accept negative values. The value must be less than or equal to the CPU limit set for the namespace. If you want to set the CPU request for more than one namespace, then ensure that the total requests of all the namespaces does not exceed the corresponding request set under sliceQuota.	Optional
memory	String	It sets the memory request total for all the pods of the namespace. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the memory limit set for the namespace. If you want to set the memory request for more than one namespace, then ensure that the total request of all the namespaces does not exceed the corresponding request set under sliceQuota.	Optional
ephemeralStorage	String	It sets the request for local ephemeral storage for all pods of the namespace. Its value is in Mi and it does not accept negative values. The value must be less than or equal to the ephemeral storage request set for the namespace. If you want to set the ephemeral storage request for more than one namespace, then ensure that the total requests of all the namespaces does not exceed the corresponding request set under sliceQuota.	Optional

defaultLimitPerContainer Parameters under namespaceQuota

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the default CPU limit per container for all the pods of a namespace. Its value is in millicores and it does not accept negative values. If the CPU limit is configured, then you must set the default CPU limit per container. This value must be less than or equal to the CPU limit set for the namespace.	Optional
memory	String	It sets the default memory limit per container for all the pods of a namespace. Its value is in Mi and it does not accept negative values. If the memory limit is configured, then you must set the default memory limit per container. This value must be less than or equal to the memory limit set for the namespace.	Optional
ephemeralStorage	String	It sets the default local ephemeral storage limit per container for all the pods of a namespace. Its value is in Mi and it does not accept negative values. If the ephemeral storage limit is configured, then you must set the default ephemeral storage limit per container. This value must be less than or equal to the ephemeral storage limit set for the namespace.	Optional

defaultRequestPerContainer Parameters under namespaceQuota

info

The defaultRequestPerContainer set at the namespace level overrides the corresponding value set under sliceQuota.

Parameter	Parameter Type	Description	Required
cpu	String	It sets the the default CPU request per container for all the pods of a namespace. Its value is in millicores and it does not accept negative values. If this parameter is not set at the slice level, then you must set it at the namespace level. This value must be less than or equal to the default limit per container and the request set for the namespace.	Optional
memory	String	It sets the default memory request for all the pods of a namespace. Its value is in Mi and it does not accept negative values. If this parameter is not set at the slice level, then you must set it at the namespace level. This value must be less than or equal to the default limit per container and the request set for the namespace.	Optional
ephemeralStorage	String	It sets the default ephemeral storage request for all the pods of the namespace. Its value is in Mi and it does not accept negative values. If this parameter is not set at the slice level, then you must set it at the namespace level. This value must be less than or equal to the default limit per container and the request set for the namespace.	Optional

Configure Quotas

clusterQuota is used as an object that contains the names of the worker clusters and namespaces that are on them. The limits can be enforced on the namespaces that are within the cluster. The limits are not set at a cluster level.

Create Resource Quota YAML

Copy and save the below slice-resource-configuration.yaml template:

apiVersion: controller.kubeslice.io/v1alpha1
kind: SliceResourceQuotaConfig
metadata:
  name: red
spec:
  sliceQuota:
    resources:
      limit:
        cpu: 1800m
        memory: 1800Mi
        podCount: 44
        ephemeralStorage: 1300Mi
      request:
        cpu: 500m
        memory: 500Mi
        ephemeralStorage: 300Mi
      defaultRequestPerContainer:
        cpu: 14m
        memory: 17Mi
        ephemeralStorage: 12Mi
  clusterQuota:
   - clusterName: cluster-1
      namespaceQuota:
        - enforceQuota: true
          namespace: namespace-1
          resources:
            limit:
              cpu: 500m
              memory: 500Mi
              podCount: 5
              ephemeralStorage: 300Mi
             request:
              cpu: 50m
              memory: 18Mi
              ephemeralStorage: 30Mi
            defaultRequestPerContainer:
              cpu: 4m
              memory: 10Mi
              ephemeralStorage: 10Mi
             defaultLimitPerContainer:
              cpu: 10m
              memory: 20Mi
              ephemeralStorage: 20Mi

Apply the Slice Quota Configuration

After creating the YAML file, apply it on the project namespace using the following command:

kubectl apply -f <slice-resource-configuration>.yaml -n <project namespace>

Validate the Slice Quota Configuration

Validate the slice quota configuration using the following command:

 kubectl get SliceResourceQuotaConfig -n <project namespace>

Example

 kubectl get SliceResourceQuotaConfig -n kubeslice-avesha

**** Expected Output****

NAME  AGE
red   10s

Edit Slice Quotas

You can edit quotas for limits and requests by:

• Editing the slice-resource-configuration.yamlfile and applying the YAML file to refresh the configuration.

Delete Slice Quotas

You can delete the slice quotas for limits by running the following command to delete the slice quotas for limits and requests:

kubectl delete SliceResourceQuotaConfig <name of the slice resource quota> -n <project namespace>