Skip to main content

Model Service

Overview​

The Model Service (ModelService) provides comprehensive model management functionality for the Kamiwaza AI Platform. Located in kamiwaza_sdk/services/models.py, this service handles model lifecycle management, including creation, deployment, file management, and configuration.

Key Features​

  • Model Management (CRUD operations)
  • Model File Management
  • Model Search and Discovery
  • Model Download Management
  • Model Configuration Management
  • Memory Usage Tracking

Model Management​

Basic Operations​

# Get a specific model
model = client.models.get_model(model_id)

# Create a new model
model = client.models.create_model(CreateModel(
    name="my-model",
    description="My custom model"
))

# List all models
models = client.models.list_models(load_files=True)

# Delete a model
client.models.delete_model(model_id)
# Search for models
models = client.models.search_models(
    query="bert",
    exact=False,
    limit=100,
    hubs_to_search=["huggingface"]
)

# Get model by repo ID
model = client.models.get_model_by_repo_id("bert-base-uncased")

External endpoint discovery and registration​

discover_external_endpoint probes a provider through Core without persisting a model or credential. register_external_endpoint registers one model with an explicit inference purpose. Registration is admin-only; discovery requires an authenticated caller and obeys Core's endpoint policy.

from kamiwaza_sdk.schemas.models.external_discovery import ExternalEndpointDiscoverRequest
from kamiwaza_sdk.schemas.models.external_registration import ExternalModelRegistration, OpenAIChatEndpoint

# Obtain provider_key from your approved secret source; never log it.
inventory = client.models.discover_external_endpoint(
    ExternalEndpointDiscoverRequest(base_url=provider_url, api_key=provider_key)
)
model = client.models.register_external_endpoint(ExternalModelRegistration(
    name="Team chat", purpose="chat",
    endpoint=OpenAIChatEndpoint(base_url=provider_url, model_id=selected_model_id),
    api_key=provider_key, credential_label="dedicated-team-chat",
))

Use the matching endpoint shape for OpenAI-compatible chat, embedding or transcription, Azure OpenAI chat/transcription, or Azure AI Foundry chat. The purpose must agree with the endpoint shape; chat-only options cannot be passed to embedding or transcription endpoints. Registered models use normal serving deployment and retirement methods.

Supply exactly one of api_key and credential_secret_urn. Inline keys become encrypted Catalog secrets; references resolve server-side. A unique credential_label isolates a newly created secret. force_replace_credentials=True rotates a shared credential for every model that references it. Deleting the last referencing model can also delete that secret, including a secret supplied by reference. Check ownership before retirement.

The existing AWS register_external_model helper remains available. Its credential accepts the existing inline models/dicts/JSON strings, legacy raw URN strings, or StoredCredentialReference(credential_secret_urn=...) from schemas.models.external_credentials. Core resolves and authorizes stored references in either wire form. The legacy helper keeps its existing model defaults; the purpose-specific non-AWS helper uses the external-model metadata produced by Core's registration wizard.

Treat failed credentialed calls as sensitive: the SDK retains server error bodies in APIError and may log them through kamiwaza_sdk.client. Prefer stored references and configure that logger appropriately when handling provider keys. Validation errors and request representations omit credential inputs; this does not redact an arbitrary server response.

Model Download Management​

Available Methods​

  • initiate_model_download(repo_id: str, quantization: str = 'q6_k') -> Dict[str, Any]: Start model download
  • check_download_status(repo_id: str) -> List[ModelDownloadStatus]: Check download progress
  • get_model_files_download_status(repo_model_id: str) -> List[ModelDownloadStatus]: Get detailed file status
# Download a model
download_info = client.models.initiate_model_download(
    repo_id="llama2-7b",
    quantization="q6_k"
)

# Check download status
status = client.models.check_download_status("llama2-7b")

Model File Management​

Available Methods​

  • delete_model_file(model_file_id: UUID) -> dict: Delete a model file
  • get_model_file(model_file_id: UUID) -> ModelFile: Get file details
  • get_model_files_by_model_id(model_id: UUID) -> List[ModelFile]: Get all files for a model
  • list_model_files() -> List[ModelFile]: List all model files
  • create_model_file(model_file: CreateModelFile) -> ModelFile: Create new model file
  • search_hub_model_files(search_request: HubModelFileSearch) -> List[ModelFile]: Search hub files
  • get_model_file_memory_usage(model_file_id: UUID) -> int: Get file memory usage
# List model files
files = client.models.list_model_files()

# Get files for specific model
model_files = client.models.get_model_files_by_model_id(model_id)

# Search hub files
files = client.models.search_hub_model_files(HubModelFileSearch(
    hub="huggingface",
    model="bert-base-uncased"
))

Model Configuration Management​

Available Methods​

  • create_model_config(config: CreateModelConfig) -> ModelConfig: Create new config
  • get_model_configs(model_id: UUID) -> List[ModelConfig]: Get all configs
  • get_model_configs_for_model(model_id: UUID, default: bool = False) -> List[ModelConfig]: Get model configs
# Create model configuration
config = client.models.create_model_config(CreateModelConfig(
    model_id=model_id,
    parameters={"temperature": 0.7}
))

# Get configurations for model
configs = client.models.get_model_configs(model_id)

Memory Usage Tracking​

Available Methods​

  • get_model_memory_usage(model_id: UUID) -> int: Get model memory usage
  • get_model_file_memory_usage(model_file_id: UUID) -> int: Get file memory usage
# Check model memory usage
memory_usage = client.models.get_model_memory_usage(model_id)

Error Handling​

The service includes built-in error handling for common scenarios:

try:
    model = client.models.get_model(model_id)
except APIError as e:
    print(f"Operation failed: {e}")

Best Practices​

  1. Always check model compatibility before downloading
  2. Monitor download status for large models
  3. Use appropriate quantization for your use case
  4. Clean up unused model files to manage storage
  5. Keep track of model configurations
  6. Monitor memory usage for large models