Data Model Design
Introduction
This document details the data model design for the Meta Agent Platform. It defines the structure, relationships, and schemas for all data stored within the platform. The data model is designed to support the core functionality of the platform while enabling future extensions for multi-modal agents, edge computing, federated collaboration, and AI-driven platform optimization.
Design Principles
The data model design adheres to the following principles:
1. Coherent Schema Design
- Use clear, consistent naming conventions
- Design for appropriate normalization
- Leverage appropriate table relationships
- Balance normalization with query performance
2. Extensibility
- Support addition of new entity types without schema redesign
- Allow for extensible properties via JSON/JSONB fields (Note: Schemas for significant JSONB fields should be defined using JSON Schema standard and documented alongside the model)
- Design for backward compatibility during evolution
- Enable multi-tenant isolation within shared schema
3. Performance Optimization
- Appropriate indexing strategy for common queries
- Selective denormalization for read-heavy operations
- Efficient storage of large objects and binary data
- Support for horizontal scaling and sharding
4. Security & Compliance
- Support for row-level security
- Data classification and sensitivity tagging
- Audit trail for all critical data changes
- Compliance with data protection regulations
5. Edge & Federated Support
- Lightweight subset for edge deployment
- Conflict resolution mechanisms for synchronization
- Schema design for secure federated queries
- Data lineage tracking across organizations
Database Technology
The Meta Agent Platform uses a hybrid database approach:
Primary Database
- PostgreSQL (v15+): Main relational database for structured data
- JSONB support for semi-structured data
- Strong transaction support
- Advanced indexing capabilities
- Rich query language
Specialized Storage
- SQLite: Lightweight database for edge deployments
- CockroachDB: Distributed SQL for federated scenarios
- Object Storage: For large binary artifacts (logs, model files) (e.g., AWS S3, MinIO - Specific choice TBD)
- Time Series DB: For metrics and observability data (optional) (e.g., InfluxDB, TimescaleDB - Specific choice TBD)
Core Data Models
1. User & Authentication Models
Users
Stores information about platform users.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| username | VARCHAR(100) | Unique username |
| VARCHAR(255) | User's email address | |
| full_name | VARCHAR(255) | User's full name |
| hashed_password | VARCHAR(255) | Securely hashed password |
| is_active | BOOLEAN | Whether the user account is active |
| is_superuser | BOOLEAN | Whether the user has superuser privileges |
| created_at | TIMESTAMP | When the user was created |
| updated_at | TIMESTAMP | When the user was last updated |
| last_login | TIMESTAMP | When the user last logged in |
| tenant_id | UUID | Reference to tenant (for multi-tenancy) |
| preferences | JSONB | User preferences and settings |
| avatar_url | VARCHAR(255) | URL to user avatar image |
Indexes: - Primary Key: id - Unique Index: username - Unique Index: email - Index: tenant_id
Roles
Defines roles for role-based access control.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(100) | Role name |
| description | TEXT | Role description |
| tenant_id | UUID | Reference to tenant |
| is_system | BOOLEAN | Whether this is a system-defined role |
| created_at | TIMESTAMP | When the role was created |
| updated_at | TIMESTAMP | When the role was last updated |
Indexes: - Primary Key: id - Unique Index: (name, tenant_id) - Index: tenant_id
User_Roles
Maps users to roles (many-to-many).
| Column | Type | Description |
|---|---|---|
| user_id | UUID | Reference to user |
| role_id | UUID | Reference to role |
| assigned_at | TIMESTAMP | When the role was assigned |
| assigned_by | UUID | Reference to user who assigned the role |
Indexes: - Primary Key: (user_id, role_id) - Index: user_id - Index: role_id
Permissions
Defines granular permissions that can be assigned to roles.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(100) | Permission name |
| description | TEXT | Permission description |
| resource | VARCHAR(100) | Resource this permission applies to (e.g., 'workflow', 'agent', 'tenant') |
| action | VARCHAR(100) | Action this permission allows (e.g., 'create', 'read', 'update', 'delete', 'execute', 'manage_users'). Proposal: Use format 'resource:action' for name, e.g., 'workflow:create'. Full list TBD. |
| is_system | BOOLEAN | Whether this is a system-defined permission |
| created_at | TIMESTAMP | When the permission was created |
Indexes: - Primary Key: id - Unique Index: name - Index: resource
Role_Permissions
Maps roles to permissions (many-to-many).
| Column | Type | Description |
|---|---|---|
| role_id | UUID | Reference to role |
| permission_id | UUID | Reference to permission |
| granted_at | TIMESTAMP | When the permission was granted |
| granted_by | UUID | Reference to user who granted the permission |
Indexes: - Primary Key: (role_id, permission_id) - Index: role_id - Index: permission_id
2. Multi-Tenancy Models
Tenants
Stores information about tenants in a multi-tenant deployment.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Tenant name |
| domain | VARCHAR(255) | Primary domain for tenant |
| status | VARCHAR(50) | Tenant status (active, suspended, etc.) |
| created_at | TIMESTAMP | When the tenant was created |
| updated_at | TIMESTAMP | When the tenant was last updated |
| settings | JSONB | Tenant-specific settings |
| plan_id | UUID | Reference to subscription plan (See Subscription_Plans table) |
| owner_id | UUID | Reference to tenant owner |
Indexes: - Primary Key: id - Unique Index: domain - Index: status
Workspaces
Defines logical workspaces within a tenant for organization.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Workspace name |
| description | TEXT | Workspace description |
| tenant_id | UUID | Reference to tenant |
| created_at | TIMESTAMP | When the workspace was created |
| updated_at | TIMESTAMP | When the workspace was last updated |
| created_by | UUID | Reference to creator user |
| icon | VARCHAR(255) | Workspace icon identifier |
| settings | JSONB | Workspace-specific settings |
Indexes: - Primary Key: id - Index: tenant_id - Unique Index: (name, tenant_id)
Workspace_Members
Maps users to workspaces (many-to-many).
| Column | Type | Description |
|---|---|---|
| workspace_id | UUID | Reference to workspace |
| user_id | UUID | Reference to user |
| role | VARCHAR(50) | Role within workspace (e.g., 'owner', 'admin', 'member'). Note: These are workspace-specific roles, distinct from the main Roles table. |
| joined_at | TIMESTAMP | When the user joined |
| invited_by | UUID | Reference to user who sent the invitation |
Indexes: - Primary Key: (workspace_id, user_id) - Index: workspace_id - Index: user_id - Index: role
3. Workflow Models
Workflows
Stores workflow definitions.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Workflow name |
| description | TEXT | Workflow description |
| definition | JSONB | Complete workflow definition (nodes, edges, properties). Note: Schema based on visual builder (React Flow) output, compatible with Temporal. Specific JSON schema TBD. |
| version | INTEGER | Workflow version number |
| status | VARCHAR(50) | Workflow status. Proposal Enum: 'draft', 'published', 'archived' |
| creator_id | UUID | Reference to creator user |
| tenant_id | UUID | Reference to tenant |
| workspace_id | UUID | Reference to workspace |
| created_at | TIMESTAMP | When the workflow was created |
| updated_at | TIMESTAMP | When the workflow was last updated |
| tags | JSONB | Array of tags for categorization |
| is_template | BOOLEAN | Whether this is a reusable template |
| orchestrator_id | VARCHAR(255) | External ID in orchestration engine (e.g., Temporal Workflow ID) |
Indexes: - Primary Key: id - Index: tenant_id - Index: workspace_id - Index: creator_id - Index: status - Index: created_at - Index: gin(tags jsonb_path_ops) - Index: (name, tenant_id)
Workflow_Versions
Tracks historical versions of workflows.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| workflow_id | UUID | Reference to workflow |
| version | INTEGER | Version number |
| definition | JSONB | Workflow definition at this version |
| created_at | TIMESTAMP | When this version was created |
| created_by | UUID | Reference to user who created this version |
| comment | TEXT | Version comment/changelog |
| status | VARCHAR(50) | Version status. Proposal Enum: 'draft', 'published' |
Indexes: - Primary Key: id - Unique Index: (workflow_id, version) - Index: workflow_id - Index: created_at
Workflow_Runs
Tracks executions of workflows.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| workflow_id | UUID | Reference to workflow |
| workflow_version | INTEGER | Version of workflow executed |
| status | VARCHAR(50) | Run status. Proposal Enum: 'queued', 'running', 'completed', 'failed', 'cancelled', 'paused', 'waiting_hitl' |
| started_at | TIMESTAMP | When the run started |
| completed_at | TIMESTAMP | When the run completed |
| inputs | JSONB | Input parameters for the run |
| outputs | JSONB | Output results from the run |
| error | TEXT | Error message if failed |
| triggered_by | UUID | Reference to user who triggered the run |
| tenant_id | UUID | Reference to tenant |
| workspace_id | UUID | Reference to workspace |
| orchestrator_run_id | VARCHAR(255) | External ID in orchestration engine (e.g., Temporal Run ID) |
| tags | JSONB | Array of tags for categorization |
| parent_run_id | UUID | Reference to parent run (for sub-workflows) |
Indexes: - Primary Key: id - Index: workflow_id - Index: status - Index: started_at - Index: tenant_id - Index: workspace_id - Index: triggered_by - Index: parent_run_id - Index: gin(tags jsonb_path_ops)
Task_Instances
Tracks execution of individual tasks within workflow runs.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| workflow_run_id | UUID | Reference to workflow run |
| node_id | VARCHAR(255) | ID of node in workflow definition |
| status | VARCHAR(50) | Task status. Proposal Enum: 'queued', 'running', 'completed', 'failed', 'cancelled', 'paused' |
| started_at | TIMESTAMP | When the task started |
| completed_at | TIMESTAMP | When the task completed |
| inputs | JSONB | Input parameters for the task |
| outputs | JSONB | Output results from the task |
| error | TEXT | Error message if failed |
| retry_count | INTEGER | Number of retries attempted |
| orchestrator_task_id | VARCHAR(255) | External ID in orchestration engine (e.g., Temporal Activity ID) |
| agent_type | VARCHAR(100) | Type of agent. Proposal Enum: 'docker', 'api', 'a2a', 'llm', 'vision', 'audio', 'sensor', 'workflow' |
| agent_config | JSONB | Configuration used for the agent (structure depends on agent_type) |
Indexes: - Primary Key: id - Index: workflow_run_id - Index: status - Index: started_at - Index: (workflow_run_id, node_id) - Index: agent_type
4. HITL Models
HITL_Tasks
Tracks human-in-the-loop tasks.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| workflow_run_id | UUID | Reference to workflow run |
| task_instance_id | UUID | Reference to task instance |
| title | VARCHAR(255) | Task title |
| description | TEXT | Task description/instructions |
| status | VARCHAR(50) | Task status. Proposal Enum: 'pending', 'assigned', 'in_progress', 'completed', 'escalated', 'expired', 'cancelled' |
| created_at | TIMESTAMP | When the task was created |
| updated_at | TIMESTAMP | When the task was last updated |
| deadline | TIMESTAMP | When the task expires |
| assignee_id | UUID | Reference to assigned user |
| assignee_role | VARCHAR(100) | Role required for assignment |
| tenant_id | UUID | Reference to tenant |
| workspace_id | UUID | Reference to workspace |
| context | JSONB | Additional context information. Note: Schema defined per HITL task type, documented separately. |
| decision_type | VARCHAR(50) | Type of decision required. Proposal Enum: 'approval', 'choice', 'form_input', 'annotation', 'free_text' |
| decision_options | JSONB | Available options for decision (e.g., for 'choice' type). Note: Schema defined per HITL task type. |
| priority | INTEGER | Task priority (1-5) |
| escalation_path | JSONB | Escalation configuration (e.g., rules, target roles/users). Note: Schema TBD. |
Indexes: - Primary Key: id - Index: workflow_run_id - Index: task_instance_id - Index: assignee_id - Index: assignee_role - Index: status - Index: priority - Index: created_at - Index: deadline - Index: tenant_id - Index: workspace_id
HITL_Decisions
Records decisions made on HITL tasks.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| hitl_task_id | UUID | Reference to HITL task |
| decision | JSONB | Decision data |
| decision_type | VARCHAR(50) | Type of decision made (matches HITL_Tasks.decision_type) |
| decided_at | TIMESTAMP | When the decision was made |
| decided_by | UUID | Reference to user who made the decision |
| comments | TEXT | Decision comments |
| attachments | JSONB | References to attached files |
Indexes: - Primary Key: id - Index: hitl_task_id - Index: decided_by - Index: decided_at
HITL_Escalations
Tracks escalations of HITL tasks.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| hitl_task_id | UUID | Reference to HITL task |
| escalated_at | TIMESTAMP | When the escalation occurred |
| escalated_by | UUID | Reference to user who escalated |
| escalated_to | UUID | Reference to user escalated to |
| escalated_to_role | VARCHAR(100) | Role escalated to |
| reason | TEXT | Escalation reason |
| status | VARCHAR(50) | Escalation status. Proposal Enum: 'pending', 'acknowledged', 'resolved', 'rejected' |
Indexes: - Primary Key: id - Index: hitl_task_id - Index: escalated_by - Index: escalated_to - Index: escalated_to_role - Index: status
5. Agent Registry Models
Agents
Stores agent definitions in the registry.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Agent name |
| description | TEXT | Agent description |
| type | VARCHAR(100) | Agent type. Proposal Enum: 'docker', 'api', 'a2a', 'llm', 'vision', 'audio', 'sensor', 'workflow' |
| configuration | JSONB | Agent configuration. Note: Structure varies by agent type, validated against input_schema. |
| version | VARCHAR(50) | Agent version (semver) |
| created_at | TIMESTAMP | When the agent was created |
| updated_at | TIMESTAMP | When the agent was last updated |
| owner_id | UUID | Reference to owner user |
| tenant_id | UUID | Reference to tenant |
| workspace_id | UUID | Reference to workspace |
| visibility | VARCHAR(50) | Visibility. Proposal Enum: 'private', 'workspace', 'tenant', 'public' |
| tags | JSONB | Array of tags for categorization |
| documentation | TEXT | Usage documentation |
| license | VARCHAR(100) | License type |
| repository_url | VARCHAR(255) | Source code repository URL |
| input_schema | JSONB | JSON Schema for agent inputs. Proposal: Use JSON Schema Draft 7 or later. |
| output_schema | JSONB | JSON Schema for agent outputs. Proposal: Use JSON Schema Draft 7 or later. |
| modalities | JSONB | Supported modalities. Proposal Enum Array: ['text', 'code', 'image', 'audio', 'video', 'sensor_tabular', 'sensor_timeseries', 'vector'] |
Indexes: - Primary Key: id - Index: owner_id - Index: tenant_id - Index: workspace_id - Index: type - Index: visibility - Index: gin(tags jsonb_path_ops) - Index: gin(modalities jsonb_path_ops) - Index: created_at - Index: (name, tenant_id)
Agent_Versions
Tracks historical versions of agents.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| agent_id | UUID | Reference to agent |
| version | VARCHAR(50) | Version string (semver) |
| configuration | JSONB | Agent configuration at this version |
| created_at | TIMESTAMP | When this version was created |
| created_by | UUID | Reference to user who created this version |
| changelog | TEXT | Version changelog |
| status | VARCHAR(50) | Version status. Proposal Enum: 'draft', 'published', 'deprecated' |
| input_schema | JSONB | JSON Schema for agent inputs (at this version) |
| output_schema | JSONB | JSON Schema for agent outputs (at this version) |
Indexes: - Primary Key: id - Unique Index: (agent_id, version) - Index: agent_id - Index: created_at - Index: status
Agent_Dependencies
Tracks dependencies between agents.
| Column | Type | Description |
|---|---|---|
| dependent_agent_id | UUID | Reference to dependent agent |
| dependency_agent_id | UUID | Reference to dependency agent |
| dependency_version | VARCHAR(50) | Required version constraint |
| dependency_type | VARCHAR(50) | Type of dependency. Proposal Enum: 'agent', 'model', 'library' |
| optional | BOOLEAN | Whether the dependency is optional |
Indexes: - Primary Key: (dependent_agent_id, dependency_agent_id) - Index: dependent_agent_id - Index: dependency_agent_id
6. Marketplace Models
Marketplace_Listings
Stores listings in the marketplace.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| title | VARCHAR(255) | Listing title |
| description | TEXT | Listing description |
| item_type | VARCHAR(50) | Type of item. Proposal Enum: 'agent', 'workflow_template', 'plugin' |
| item_id | UUID | Reference to item (polymorphic, see note below) |
| price | DECIMAL | Price (0 for free) |
| currency | VARCHAR(3) | Currency code (e.g., 'USD') |
| pricing_model | VARCHAR(50) | Pricing model. Proposal Enum: 'free', 'one_time', 'subscription_monthly', 'subscription_yearly' |
| created_at | TIMESTAMP | When the listing was created |
| updated_at | TIMESTAMP | When the listing was last updated |
| publisher_id | UUID | Reference to publisher user |
| tenant_id | UUID | Reference to publisher tenant |
| status | VARCHAR(50) | Listing status. Proposal Enum: 'draft', 'pending_review', 'published', 'rejected', 'suspended', 'archived' |
| featured | BOOLEAN | Whether this is a featured listing |
| tags | JSONB | Array of tags for categorization |
| screenshots | JSONB | Array of screenshot URLs |
| video_url | VARCHAR(255) | Demo video URL |
| downloads | INTEGER | Download count |
| average_rating | DECIMAL | Average rating (1-5) |
| rating_count | INTEGER | Number of ratings |
| license_terms | TEXT | License terms and conditions |
| support_url | VARCHAR(255) | Support website URL |
| categories | JSONB | Array of category IDs (See Marketplace_Categories table) |
| quality_score | INTEGER | System-calculated quality score. Note: Calculation based on tests, ratings, usage, compliance. Algorithm TBD. |
| verified | BOOLEAN | Whether this listing is verified |
| compliance_badges | JSONB | Array of compliance certifications |
Note on item_id: This is a polymorphic relationship. Consider adding specific nullable FK columns like agent_id UUID NULL REFERENCES Agents(id), workflow_template_id UUID NULL REFERENCES Workflows(id) and a CHECK constraint ensuring only one is non-null based on item_type.
Indexes: - Primary Key: id - Index: item_id - Index: publisher_id - Index: tenant_id - Index: status - Index: item_type - Index: created_at - Index: featured - Index: downloads - Index: average_rating - Index: quality_score - Index: verified - Index: gin(tags jsonb_path_ops) - Index: gin(categories jsonb_path_ops) - Index: gin(compliance_badges jsonb_path_ops)
Marketplace_Categories
Defines categories for marketplace items.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(100) | Category name |
| description | TEXT | Category description |
| parent_category_id | UUID | Reference to parent category (for hierarchy) |
| created_at | TIMESTAMP | When the category was created |
Indexes: - Primary Key: id - Unique Index: name - Index: parent_category_id
Marketplace_Listing_Categories
Maps listings to categories (many-to-many).
| Column | Type | Description |
|---|---|---|
| listing_id | UUID | Reference to marketplace listing |
| category_id | UUID | Reference to marketplace category |
Indexes: - Primary Key: (listing_id, category_id) - Index: listing_id - Index: category_id
Marketplace_Categories
Defines categories for marketplace items.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(100) | Category name |
| description | TEXT | Category description |
| parent_category_id | UUID | Reference to parent category (for hierarchy) |
| created_at | TIMESTAMP | When the category was created |
Indexes: - Primary Key: id - Unique Index: name - Index: parent_category_id
Marketplace_Listing_Categories
Maps listings to categories (many-to-many).
| Column | Type | Description |
|---|---|---|
| listing_id | UUID | Reference to marketplace listing |
| category_id | UUID | Reference to marketplace category |
Indexes: - Primary Key: (listing_id, category_id) - Index: listing_id - Index: category_id
Marketplace_Reviews
Stores user reviews for marketplace items.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| listing_id | UUID | Reference to marketplace listing |
| user_id | UUID | Reference to reviewer user |
| rating | INTEGER | Rating (1-5) |
| title | VARCHAR(255) | Review title |
| content | TEXT | Review content |
| created_at | TIMESTAMP | When the review was created |
| updated_at | TIMESTAMP | When the review was last updated |
| helpful_votes | INTEGER | Number of helpful votes |
| verified_purchase | BOOLEAN | Whether reviewer has purchased the item |
| version_used | VARCHAR(50) | Version being reviewed |
Indexes: - Primary Key: id - Index: listing_id - Index: user_id - Index: rating - Index: created_at - Index: helpful_votes
Marketplace_Transactions
Records marketplace purchases and subscriptions.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| listing_id | UUID | Reference to marketplace listing |
| buyer_id | UUID | Reference to buyer user |
| buyer_tenant_id | UUID | Reference to buyer tenant |
| seller_id | UUID | Reference to seller user |
| seller_tenant_id | UUID | Reference to seller tenant |
| transaction_type | VARCHAR(50) | Type. Proposal Enum: 'purchase', 'subscription_start', 'subscription_renewal', 'refund' |
| amount | DECIMAL | Transaction amount |
| currency | VARCHAR(3) | Currency code (e.g., 'USD') |
| status | VARCHAR(50) | Transaction status. Proposal Enum: 'pending', 'completed', 'failed', 'refunded' |
| created_at | TIMESTAMP | When the transaction occurred |
| payment_method | VARCHAR(50) | Payment method used (e.g., 'stripe', 'paypal') |
| payment_reference | VARCHAR(255) | External payment reference |
| subscription_id | UUID | Reference to subscription record |
Indexes: - Primary Key: id - Index: listing_id - Index: buyer_id - Index: buyer_tenant_id - Index: seller_id - Index: seller_tenant_id - Index: transaction_type - Index: status - Index: created_at - Index: subscription_id
Marketplace_Subscriptions
Tracks active subscriptions to marketplace items.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| listing_id | UUID | Reference to marketplace listing |
| subscriber_id | UUID | Reference to subscriber user |
| subscriber_tenant_id | UUID | Reference to subscriber tenant |
| plan_id | VARCHAR(255) | Subscription plan identifier (defined by publisher/marketplace) |
| status | VARCHAR(50) | Subscription status. Proposal Enum: 'active', 'past_due', 'cancelled', 'expired' |
| started_at | TIMESTAMP | When the subscription started |
| expires_at | TIMESTAMP | When the subscription expires (for fixed-term) |
| renewal_date | TIMESTAMP | Next renewal date |
| amount | DECIMAL | Recurring amount |
| currency | VARCHAR(3) | Currency code |
| billing_frequency | VARCHAR(50) | Billing frequency. Proposal Enum: 'monthly', 'yearly', 'one_time' |
| auto_renew | BOOLEAN | Whether to auto-renew |
| payment_method | VARCHAR(50) | Payment method used (e.g., 'stripe', 'paypal') |
| external_subscription_id | VARCHAR(255) | External subscription reference (e.g., Stripe Subscription ID) |
Subscription_Plans
Defines subscription plans offered by tenants/publishers (referenced by Tenants.plan_id and potentially Marketplace_Subscriptions.plan_id).
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Plan name |
| description | TEXT | Plan description |
| tenant_id | UUID | Reference to tenant offering the plan (NULL for platform plans) |
| features | JSONB | Features included in the plan (e.g., usage limits, access levels) |
| price | DECIMAL | Price per billing frequency |
| currency | VARCHAR(3) | Currency code |
| billing_frequency | VARCHAR(50) | Billing frequency. Proposal Enum: 'monthly', 'yearly' |
| is_active | BOOLEAN | Whether the plan is currently offered |
| created_at | TIMESTAMP | When the plan was created |
| updated_at | TIMESTAMP | When the plan was last updated |
Indexes: - Primary Key: id - Index: tenant_id - Index: is_active
Indexes: - Primary Key: id - Index: listing_id - Index: subscriber_id - Index: subscriber_tenant_id - Index: status - Index: renewal_date
7. Multi-Modal Agent Models
Vision_Models
Stores vision model configurations.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Model name |
| description | TEXT | Model description |
| model_type | VARCHAR(100) | Type of model |
| capabilities | JSONB | Array of capabilities |
| provider | VARCHAR(100) | Model provider |
| api_endpoint | VARCHAR(255) | API endpoint for hosted models |
| container_image | VARCHAR(255) | Container image for local models |
| input_formats | JSONB | Supported input formats |
| output_formats | JSONB | Supported output formats |
| created_at | TIMESTAMP | When the model was created |
| updated_at | TIMESTAMP | When the model was last updated |
| owner_id | UUID | Reference to owner user |
| tenant_id | UUID | Reference to tenant |
| version | VARCHAR(50) | Model version |
| settings | JSONB | Model settings and parameters |
| resource_requirements | JSONB | CPU/GPU/memory requirements |
Indexes: - Primary Key: id - Index: owner_id - Index: tenant_id - Index: model_type - Index: provider - Index: gin(capabilities jsonb_path_ops) - Index: (name, tenant_id)
Audio_Models
Stores audio model configurations.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Model name |
| description | TEXT | Model description |
| model_type | VARCHAR(100) | Type of model (speech-to-text, text-to-speech, etc.) |
| capabilities | JSONB | Array of capabilities |
| provider | VARCHAR(100) | Model provider |
| api_endpoint | VARCHAR(255) | API endpoint for hosted models |
| container_image | VARCHAR(255) | Container image for local models |
| supported_languages | JSONB | Array of supported languages |
| input_formats | JSONB | Supported input formats |
| output_formats | JSONB | Supported output formats |
| created_at | TIMESTAMP | When the model was created |
| updated_at | TIMESTAMP | When the model was last updated |
| owner_id | UUID | Reference to owner user |
| tenant_id | UUID | Reference to tenant |
| version | VARCHAR(50) | Model version |
| settings | JSONB | Model settings and parameters |
| resource_requirements | JSONB | CPU/GPU/memory requirements |
Indexes: - Primary Key: id - Index: owner_id - Index: tenant_id - Index: model_type - Index: provider - Index: gin(capabilities jsonb_path_ops) - Index: gin(supported_languages jsonb_path_ops) - Index: (name, tenant_id)
Sensor_Models
Stores sensor data processing model configurations.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Model name |
| description | TEXT | Model description |
| model_type | VARCHAR(100) | Type of model |
| sensor_types | JSONB | Types of sensors supported |
| protocols | JSONB | Supported protocols |
| data_formats | JSONB | Supported data formats |
| capabilities | JSONB | Array of capabilities |
| provider | VARCHAR(100) | Model provider |
| api_endpoint | VARCHAR(255) | API endpoint for hosted models |
| container_image | VARCHAR(255) | Container image for local models |
| created_at | TIMESTAMP | When the model was created |
| updated_at | TIMESTAMP | When the model was last updated |
| owner_id | UUID | Reference to owner user |
| tenant_id | UUID | Reference to tenant |
| version | VARCHAR(50) | Model version |
| settings | JSONB | Model settings and parameters |
| resource_requirements | JSONB | CPU/GPU/memory requirements |
Indexes: - Primary Key: id - Index: owner_id - Index: tenant_id - Index: model_type - Index: provider - Index: gin(capabilities jsonb_path_ops) - Index: gin(sensor_types jsonb_path_ops) - Index: gin(protocols jsonb_path_ops) - Index: (name, tenant_id)
8. Edge Computing Models
Edge_Devices
Stores information about edge devices.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Device name |
| description | TEXT | Device description |
| device_type | VARCHAR(100) | Type of edge device |
| status | VARCHAR(50) | Device status. Proposal Enum: 'registered', 'online', 'offline', 'error', 'decommissioned' |
| last_seen | TIMESTAMP | When device last connected |
| registration_date | TIMESTAMP | When device was registered |
| owner_id | UUID | Reference to owner user |
| tenant_id | UUID | Reference to tenant |
| workspace_id | UUID | Reference to workspace |
| capabilities | JSONB | Device capabilities |
| system_info | JSONB | OS, CPU, memory, etc. |
| network_info | JSONB | Connectivity information |
| location | JSONB | Geographic location |
| api_key | VARCHAR(255) | Device API key (hashed) |
| settings | JSONB | Device-specific settings |
| tags | JSONB | Array of tags for categorization |
| version | VARCHAR(50) | Edge runtime version |
Indexes: - Primary Key: id - Index: owner_id - Index: tenant_id - Index: workspace_id - Index: device_type - Index: status - Index: last_seen - Index: registration_date - Index: gin(capabilities jsonb_path_ops) - Index: gin(tags jsonb_path_ops) - Index: (name, tenant_id)
Edge_Deployments
Tracks deployments to edge devices.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| device_id | UUID | Reference to edge device |
| workflow_id | UUID | Reference to deployed workflow |
| workflow_version | INTEGER | Version of deployed workflow |
| status | VARCHAR(50) | Deployment status. Proposal Enum: 'pending', 'deploying', 'active', 'failed', 'inactive', 'deleting' |
| deployed_at | TIMESTAMP | When the deployment occurred |
| deployed_by | UUID | Reference to user who deployed |
| active | BOOLEAN | Whether deployment is active |
| configuration | JSONB | Deployment configuration |
| last_sync | TIMESTAMP | When last synchronized |
| sync_status | VARCHAR(50) | Synchronization status. Proposal Enum: 'synced', 'syncing', 'pending', 'error' |
| error | TEXT | Error message if failed |
| metrics | JSONB | Performance metrics |
Indexes: - Primary Key: id - Index: device_id - Index: workflow_id - Index: status - Index: deployed_at - Index: last_sync - Index: active
Edge_Telemetry
Stores telemetry data from edge devices.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| device_id | UUID | Reference to edge device |
| timestamp | TIMESTAMP | When telemetry was collected |
| metrics | JSONB | Performance metrics |
| status | JSONB | Component status information |
| logs | TEXT | Aggregated log entries |
| errors | JSONB | Error information |
| network_stats | JSONB | Network performance data |
| resource_usage | JSONB | CPU, memory, disk usage |
| battery_status | JSONB | Battery information (if applicable) |
| received_at | TIMESTAMP | When telemetry was received by server |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: device_id - Index: timestamp - Index: tenant_id - Index: received_at - Index: (device_id, timestamp)
Edge_Sync_Queue
Manages synchronization queue for edge devices.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| device_id | UUID | Reference to edge device |
| direction | VARCHAR(10) | Sync direction. Proposal Enum: 'up', 'down' |
| data_type | VARCHAR(50) | Type of data to sync (e.g., 'workflow_run', 'telemetry', 'config') |
| priority | INTEGER | Sync priority (lower value = higher priority) |
| status | VARCHAR(50) | Queue item status. Proposal Enum: 'queued', 'processing', 'completed', 'failed' |
| created_at | TIMESTAMP | When item was added to queue |
| last_attempt | TIMESTAMP | When sync was last attempted |
| attempts | INTEGER | Number of attempts |
| data_reference | TEXT | Reference to data to sync. Note: Could be URI (e.g., sqlite://local.db?table=X&id=Y) or object storage ref. Mechanism TBD. |
| size | INTEGER | Size of data in bytes |
Indexes: - Primary Key: id - Index: device_id - Index: direction - Index: data_type - Index: status - Index: priority - Index: created_at - Index: (device_id, status, priority)
9. Federated Collaboration Models
Federated_Organizations
Stores information about federated organizations.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Organization name |
| description | TEXT | Organization description |
| domain | VARCHAR(255) | Organization domain |
| api_endpoint | VARCHAR(255) | API endpoint for organization |
| status | VARCHAR(50) | Federation status. Proposal Enum: 'pending_invite', 'active', 'suspended', 'revoked' |
| trust_level | VARCHAR(50) | Trust level. Proposal Enum: 'low', 'medium', 'high', 'verified' |
| public_key | TEXT | Organization public key |
| created_at | TIMESTAMP | When organization was created |
| updated_at | TIMESTAMP | When organization was last updated |
| verified | BOOLEAN | Whether organization is verified |
| tenant_id | UUID | Reference to tenant |
| capabilities | JSONB | Supported capabilities |
| settings | JSONB | Federation settings |
| metadata | JSONB | Additional metadata |
Indexes: - Primary Key: id - Index: tenant_id - Index: status - Index: trust_level - Index: verified - Index: gin(capabilities jsonb_path_ops) - Unique Index: domain
Federated_Workflows
Stores information about cross-organization workflows.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Workflow name |
| description | TEXT | Workflow description |
| participants | JSONB | Participating organizations |
| owner_org_id | UUID | Reference to owner organization |
| status | VARCHAR(50) | Workflow status. Proposal Enum: 'draft', 'proposed', 'active', 'completed', 'failed', 'archived' |
| created_at | TIMESTAMP | When workflow was created |
| updated_at | TIMESTAMP | When workflow was last updated |
| definition | JSONB | Cross-org workflow definition. Note: Schema based on federation protocol, TBD. |
| access_controls | JSONB | Access control policies |
| data_policies | JSONB | Data handling policies |
| tenant_id | UUID | Reference to tenant |
| version | INTEGER | Workflow version |
| tags | JSONB | Array of tags for categorization |
Indexes: - Primary Key: id - Index: owner_org_id - Index: status - Index: created_at - Index: tenant_id - Index: gin(participants jsonb_path_ops) - Index: gin(tags jsonb_path_ops) - Index: (name, tenant_id)
Secure_Computations
Tracks secure multi-party computations.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Computation name |
| description | TEXT | Computation description |
| computation_type | VARCHAR(100) | Type of computation |
| participants | JSONB | Participating organizations |
| initiator_org_id | UUID | Reference to initiator organization |
| status | VARCHAR(50) | Computation status. Proposal Enum: 'pending', 'running', 'completed', 'failed' |
| created_at | TIMESTAMP | When computation was created |
| completed_at | TIMESTAMP | When computation completed |
| algorithm | VARCHAR(100) | Computation algorithm |
| parameters | JSONB | Computation parameters |
| result_access | JSONB | Access controls for results |
| audit_trail | JSONB | Audit information |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: initiator_org_id - Index: computation_type - Index: status - Index: created_at - Index: tenant_id - Index: gin(participants jsonb_path_ops)
Federated_Learning_Tasks
Tracks federated learning activities.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| name | VARCHAR(255) | Task name |
| description | TEXT | Task description |
| model_type | VARCHAR(100) | Type of model |
| participants | JSONB | Participating organizations |
| initiator_org_id | UUID | Reference to initiator organization |
| status | VARCHAR(50) | Task status. Proposal Enum: 'pending', 'running', 'aggregating', 'completed', 'failed' |
| created_at | TIMESTAMP | When task was created |
| updated_at | TIMESTAMP | When task was last updated |
| aggregation_method | VARCHAR(100) | Method for combining models |
| parameters | JSONB | Learning parameters |
| rounds | INTEGER | Number of training rounds |
| current_round | INTEGER | Current round number |
| model_uri | VARCHAR(255) | URI to model |
| metrics | JSONB | Performance metrics |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: initiator_org_id - Index: model_type - Index: status - Index: created_at - Index: tenant_id - Index: gin(participants jsonb_path_ops) - Index: (name, tenant_id)
10. AI-Driven Platform Models
Workflow_Optimizations
Stores AI-generated workflow optimization suggestions.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| workflow_id | UUID | Reference to workflow |
| suggested_at | TIMESTAMP | When suggestion was generated |
| type | VARCHAR(100) | Type of optimization |
| description | TEXT | Description of optimization |
| suggestions | JSONB | Detailed optimization suggestions. Note: Schema defined by AI optimization process. |
| impact_estimate | JSONB | Estimated performance impact. Note: Schema defined by AI optimization process. |
| applied | BOOLEAN | Whether suggestion was applied |
| applied_at | TIMESTAMP | When suggestion was applied |
| applied_by | UUID | Reference to user who applied |
| performance_impact | JSONB | Actual performance impact. Note: Schema defined by AI optimization process. |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: workflow_id - Index: suggested_at - Index: type - Index: applied - Index: tenant_id
Anomaly_Detections
Records detected anomalies in platform operation.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| entity_type | VARCHAR(50) | Type of entity (workflow, agent, device) |
| entity_id | UUID | Reference to entity |
| detected_at | TIMESTAMP | When anomaly was detected |
| anomaly_type | VARCHAR(100) | Type of anomaly |
| severity | VARCHAR(50) | Anomaly severity |
| description | TEXT | Description of anomaly |
| data | JSONB | Anomaly details and context. Note: Schema defined by anomaly detection process. |
| status | VARCHAR(50) | Resolution status. Proposal Enum: 'new', 'acknowledged', 'investigating', 'resolved', 'ignored' |
| resolved_at | TIMESTAMP | When anomaly was resolved |
| resolved_by | UUID | Reference to user who resolved |
| resolution_steps | TEXT | Steps taken to resolve |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: entity_type - Index: entity_id - Index: detected_at - Index: anomaly_type - Index: severity - Index: status - Index: tenant_id - Index: (entity_type, entity_id)
Self_Healing_Actions
Tracks automated recovery actions.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| entity_type | VARCHAR(50) | Type of entity (workflow, agent, device) |
| entity_id | UUID | Reference to entity |
| triggered_at | TIMESTAMP | When action was triggered |
| action_type | VARCHAR(100) | Type of action |
| description | TEXT | Description of action |
| initiator | VARCHAR(50) | What initiated the action |
| anomaly_id | UUID | Reference to anomaly |
| success | BOOLEAN | Whether action was successful |
| result | TEXT | Action result |
| error | TEXT | Error if failed |
| duration_ms | INTEGER | Action duration in milliseconds |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: entity_type - Index: entity_id - Index: triggered_at - Index: action_type - Index: success - Index: anomaly_id - Index: tenant_id
Performance_Analytics
Stores AI-generated performance analytics.
| Column | Type | Description |
|---|---|---|
| id | UUID | Primary key |
| entity_type | VARCHAR(50) | Type of entity (workflow, agent, device) |
| entity_id | UUID | Reference to entity |
| period_start | TIMESTAMP | Start of analysis period |
| period_end | TIMESTAMP | End of analysis period |
| metrics | JSONB | Performance metrics. Note: Schema defined by analytics process. |
| insights | JSONB | AI-generated insights. Note: Schema defined by analytics process. |
| recommendations | JSONB | Improvement recommendations. Note: Schema defined by analytics process. |
| trends | JSONB | Identified trends. Note: Schema defined by analytics process. |
| comparisons | JSONB | Comparative analysis. Note: Schema defined by analytics process. |
| generated_at | TIMESTAMP | When analysis was generated |
| tenant_id | UUID | Reference to tenant |
Indexes: - Primary Key: id - Index: entity_type - Index: entity_id - Index: period_start - Index: period_end - Index: generated_at - Index: tenant_id - Index: (entity_type, entity_id, period_start, period_end)
Database Relationships
The following diagram illustrates the key relationships between data models:
Users 1──* User_Roles *──1 Roles *──* Role_Permissions 1──* Permissions
|
|
1
|
*
Tenants 1──* Workspaces *──* Workspace_Members *──1 Users
|
|
1
|
*
Workflows 1──* Workflow_Versions
| |
| |
1 1
| |
* *
Workflow_Runs *┬1 Workflow_Versions
| |
1 1
| |
* *
Task_Instances 1┬* HITL_Tasks 1──* HITL_Decisions
| |
| 1
| |
| *
| HITL_Escalations
|
1
|
*
Agents 1──* Agent_Versions
| |
| |
| *
| |
| *
1 Agent_Dependencies
|
|
1
|
*
Marketplace_Listings 1──* Marketplace_Reviews
|
|
1
|
*
Marketplace_Transactions *──1 Marketplace_Subscriptions
|
|
*
|
1
Agents
Note: Diagram needs updating to include relationships for Multi-Modal Models, Edge Devices/Deployments, Federated Entities, AI Platform entities, Subscription Plans, and Marketplace Categories.
Multi-Tenancy Strategy
The Meta Agent Platform uses a multi-tenant database strategy with these key characteristics:
- Tenant Identifier Column: Most tables include a
tenant_idcolumn to segregate data. - Row-Level Security: PostgreSQL row-level security policies restrict data access based on tenant.
- Connection Pooling: Tenant-aware connection pools ensure proper data isolation.
- Schema Versioning: All tenants share the same schema version for consistency.
- Query Optimization: Indexes on tenant columns optimize tenant-specific queries.
Edge Database Strategy
For edge deployments, the platform uses:
- SQLite Local Database: Lightweight storage on edge devices
- Schema Subset: Simplified schema containing only necessary tables
- Synchronization Protocol: Efficient bi-directional sync with conflict resolution. Note: Specific protocol and conflict resolution strategy (e.g., CRDTs, last-write-wins) TBD.
- Offline Operation: Full functionality without continuous connectivity
- Storage Optimization: Automatic cleanup of historical data based on relevance
Data Security Measures
The database design incorporates several security measures:
- Encryption at Rest: Database files and backups are encrypted
- Sensitive Data Handling: Passwords and secrets are hashed/encrypted
- Audit Trails: Comprehensive logging of data changes
- Principle of Least Privilege: Row-level security enforces minimal access
- Data Classification: Tagging for GDPR, HIPAA, PCI-DSS compliance
- Secure Multi-Party Computation: Privacy-preserving cross-org data processing. Note: Specific fields/tables involved (e.g.,
Federated_Learning_Tasks,Secure_Computations) TBD during feature implementation. - Homomorphic Encryption: Computing on encrypted data without exposure. Note: Specific application points TBD.
Conclusion
The data model design outlined in this document provides a comprehensive foundation for the Meta Agent Platform. It supports the core functionality while enabling future expansion for multi-modal agents, edge computing, federated collaboration, and AI-driven platform optimization.
The design balances normalization with performance, security with flexibility, and current requirements with future extensibility. It leverages PostgreSQL's advanced features while providing strategies for edge and federated scenarios. This version incorporates clarifications and proposals based on the broader design documentation, addressing many initial open questions.
As the platform evolves, this data model will be extended and refined through careful schema migrations to ensure backward compatibility and minimal disruption to existing data. Areas marked as TBD or Proposal require further definition during detailed implementation phases.