Data Ingestion

Introduction

This guide explains what your organization should provide so Novo AI can integrate with your existing production systems. It is vendor-neutral and applies to ERP, MES, BDE/shop-floor data collection, database, API, and file-based environments.

A successful integration requires more than field names. Before implementation, identify the authoritative source systems, confirm stable identifiers, define units and timestamp rules, provide representative sample data, and agree how Novo AI will access or exchange the data.

Supported integration paths

Customer-side readiness checklist

• Identify the ERP planning source and the table, view, API, or export containing planned production operations.
• Identify the machine/resource master and confirm the stable machine/resource code used across systems.
• Confirm the production-site timezone and daylight-saving-time behavior.
• Confirm quantity units such as pcs, kg, or m, and the time units used by the source system.
• Confirm whether planned production time applies per unit or to the complete order quantity.
• Provide representative test data containing multiple orders, operations, machines, and edge cases.
• If existing BDE/MES intervals are in scope, provide the state/activity model and order/resource references.
• If Novo AI data will be written back, provide the target interface/table and required authentication/network access.
• Provide technical and production-planning contacts who can validate mappings and test results.

Connection methods

Novo AI supports several standard ways to connect to your ERP or BDE/MES system. The right method depends on what your systems already expose and your internal security policies — in every case, Novo AI only needs read-only access for the planning and BDE/MES import flows described in this guide.

Example: read-only integration table

If you choose the direct-database or single-table/view option, the fastest path is to expose the planning fields already defined in Required ERP -> Novo AI planning fields as one dedicated table or view.

CREATE TABLE novoai_planning_export (
    order_id                          VARCHAR(64)               NOT NULL,
    customer_code                     VARCHAR(64),
    customer_name                     VARCHAR(255),
    order_tags                        VARCHAR(255),
    product_id                        VARCHAR(64)               NOT NULL,
    planned_quantity                  DECIMAL(18,3)              NOT NULL,
    quantity_unit                     VARCHAR(16)                NOT NULL,
    planned_start                     TIMESTAMP WITH TIME ZONE   NOT NULL,
    planned_end                       TIMESTAMP WITH TIME ZONE   NOT NULL,
    operation_sequence                VARCHAR(32)                NOT NULL,
    operation_description             VARCHAR(255),
    planned_preparation_time_seconds  INTEGER,
    planned_production_time_seconds   INTEGER,
    resource_code                     VARCHAR(64)                NOT NULL,
    resource_name                     VARCHAR(255),
    PRIMARY KEY (order_id, operation_sequence, resource_code)
);

Then create a dedicated account for Novo AI and grant it read-only access to just that table:

CREATE USER novoai_reader WITH PASSWORD '...';
GRANT SELECT ON novoai_planning_export TO novoai_reader;

This mirrors the uniqueness rule already used for planning records (order_id + operation_sequence + resource_code) and keeps Novo AI's access limited to exactly the columns and rows required — nothing else in your ERP schema is exposed. The same pattern applies to a read-only view over existing ERP tables if you would rather not duplicate data into a new table.

General data requirements

These principles apply regardless of whether data comes from an ERP, MES/BDE system, database interface, API, or spreadsheet. Most integration issues originate from ambiguous identifiers, inconsistent units, or timestamp interpretation rather than from the transport technology itself.

Stable identifiers

Identifiers used for matching must remain stable over time. Display names may change; matching keys should not. Treat identifiers as strings even when they contain digits only because leading zeros, prefixes, suffixes, and revisions may be meaningful.

Accepted logical data types

Timestamp and timezone rules

For automated interfaces, timezone-aware timestamps are strongly preferred.

Preferred: 2026-07-29T08:00:00+02:00
UTC form:  2026-07-29T06:00:00Z
Avoid:     07/29/26 08:00

If the source provides only local date/time values, the production-site timezone must be explicitly agreed during setup.

Quantities, units, and durations

• Every planned quantity must have a unit, for example planned_quantity = 250 with quantity_unit = pcs.
• Canonical duration fields ending in _seconds use non-negative integer seconds. For example, 900 means 15 minutes.
• Do not send 00:15:00 into a field documented as integer seconds unless the interface explicitly defines a duration-string format.
• For machine-readable CSV, use a dot as the decimal separator unless another file format is explicitly agreed.
• Do not assume arbitrary unit conversion. Source units and required normalization must be agreed during mapping.

Empty value vs. zero

A blank or null value and 0 have different meanings:

• Blank / null: the source did not provide the value.
• 0: the value is known and is actually zero.

Do not automatically replace unknown optional numeric values with zero. The machine-state export is a deliberate exception: non-applicable state-duration columns are intentionally set to 0 for each interval.

Recommended file hygiene

For temporary Excel/CSV imports:

• Use one header row and one planned operation per data row.
• Avoid merged cells, hidden semantic information in formatting, and formulas that depend on external workbooks.
• For CSV, agree the delimiter and encoding; UTF-8 is recommended for international text.
• Keep identifier columns as text to preserve leading zeros.
• Use one consistent timestamp representation per import.

Direct ERP integration

For a direct ERP integration, Novo AI reads planned production and order information from the customer system. One record represents one planned operation or production step assigned to a machine/resource. Read-only access to the planning source is sufficient.

Required ERP -> Novo AI planning fields

Production-time semantics

planned_production_time_seconds is only meaningful when both systems agree whether it represents the complete order quantity or one individual unit.

Where the source ERP can provide it, planned_production_time_basis is strongly recommended with values such as TOTAL_ORDER or PER_UNIT. It remains an extension field unless specifically configured as mandatory.

Optional planning fields

Recommended industrial planning attributes

These fields are useful extensions when they already exist in the customer source system or help resolve ambiguity. They are not universal mandatory requirements.

Example planning record

order_id: PO-2026-004582
customer_code: CUST-0042
customer_name: Meyer Automotive GmbH
order_tags:
  - urgent
  - export
product_id: MAT-0004711
product_description: Aluminium pump housing 120 mm
planned_quantity: 250
quantity_unit: pcs
planned_start: 2026-07-29T08:00:00+02:00
planned_end: 2026-07-29T11:00:00+02:00
operation_sequence: A10
operation_description: CNC milling
planned_preparation_time_seconds: 900
planned_production_time_seconds: 7200
planned_production_time_basis: TOTAL_ORDER
resource_code: MILL-12
resource_name: 5-Axis Milling Machine 12
priority: 50
plant_code: DE-BER-01
revision: REV-C
order_date: 2026-07-20
delivery_date: 2026-08-05
operation_notes: Use fixture F-12

Optional fields may be omitted when unavailable. The example demonstrates a complete and unambiguous record; it does not imply that every optional field must exist in every ERP system.

ERP + existing BDE/MES integration

Existing shop-floor production data may come from a BDE (Betriebsdatenerfassung), MES, machine-data platform, or another production data collection system. If these systems already record machine, activity, or process intervals, those intervals can optionally be imported into Novo AI and displayed alongside Novo AI-detected machine data.

Minimum interval data

Matching logic

Novo AI primarily uses order_id + resource_code to associate an imported interval with the corresponding ERP-planned order and the correct machine. resource_name is descriptive and should not be used as the primary matching key.

Example activity/state values

Customer-specific state IDs and names can be mapped during integration. Customers do not need to rename existing MES/BDE statuses to these exact examples.

Recommended BDE/MES enrichment fields

These fields are optional enrichments unless specifically agreed for a project:

• operation_id or operation_sequence - improves matching when several operations exist for the same order/resource.
• reason_id / reason_name - supports downtime analysis and reason mapping.
• good_quantity / scrap_quantity - useful when the source already records production counts and quality results.
• shift_id - useful for shift-level reporting where an explicit shift model exists.
• source_system - useful when intervals can originate from multiple BDE/MES sources.

Temporary ERP import via Excel/CSV

When a direct ERP connection is not yet available, planned production data can be provided through an Excel/CSV upload. This allows planning records and field semantics to be validated before the automated interface is implemented.

Upload template column reference

Combined vs. split planned timestamps

The template supports two alternative representations of the same planning times.

planned_start = 2026-03-01 06:00:00
planned_end   = 2026-03-01 08:00:00

planned_start_date = 2026-03-01
planned_start_time = 06:00:00
planned_end_date   = 2026-03-01
planned_end_time   = 08:00:00

Temporary-import time fields

planned_preparation_time and planned_production_time are numeric values, but their source unit is not encoded in the column name. Do not assume seconds. The unit must be agreed for the specific customer export and normalized where required.

Example if the source unit is minutes:
planned_preparation_time = 15

Canonical value after normalization:
planned_preparation_time_seconds = 900

The production-time value also requires a defined basis: either the complete planned order quantity or one individual unit. Both the unit and time basis must be unambiguous before the data is used for planning calculations.

Example CSV row

order,product,count,quantity_unit,planned_start,planned_end,customer_name,customer_code,priority,operation_sequence,operation_description,planned_preparation_time,planned_production_time,machine_resource,tags
SO-3001,PART-A,500,pcs,2026-03-01 06:00:00,2026-03-01 08:00:00,Acme Ltd,CUST-ACME,50,A10,Cutting,15,120,MACHINE-12,"Urgent, Export"

For a production rollout, provide several sample rows containing multiple orders, multiple operations for one order, different machines, and at least one example with optional fields.

Access, connectivity, and implementation inputs

The data dictionary defines what information is exchanged. The following customer-side inputs are also needed to implement the connection reliably. Exact technology choices depend on the customer environment and must be agreed during the project — see Connection methods above for the supported transport options.

Source and target system information

Security and network details

Provide or confirm:

• Connection method and environment: development/test/production endpoints where applicable.
• Authentication method and credential-management process.
• VPN, firewall, IP allowlisting, proxy, or certificate requirements.
• Least-privilege access: read-only for source planning/BDE data wherever sufficient; write capability only for agreed export targets.
• Credential rotation and an operational contact for connectivity incidents.

Mapping decisions to agree

• Which source field maps to each canonical order, product, customer, operation, and resource field?
• What is the authoritative resource_code used for machine matching?
• What timezone applies to local timestamps?
• Which quantity units can appear, and are any conversions required?
• What unit is used for setup and production times in the source?
• Is planned production time PER_UNIT or TOTAL_ORDER?
• How does ERP priority work: higher number = higher priority, or the reverse?
• How should customer-specific BDE/MES state IDs/names map to the desired state model?
• How should corrections, deleted orders, rescheduled operations, and renamed resources be represented?
• What is the uniqueness/deduplication rule for repeated imports or exported intervals?

Representative test data to provide

Provide representative examples covering:

• At least one order with multiple sequential operations.
• Operations assigned to different machines/resources.
• Numeric-looking identifiers with leading zeros, if they exist in production.
• An order containing optional notes, tags, and priority fields.
• A quantity using a non-piece unit such as kg or m, if applicable.
• A rescheduled operation or changed planned time, if the source supports replanning.
• If BDE/MES is included: production, setup, interruption, and maintenance intervals.
• If write-back is included: a non-production test target where interval writes can be verified safely.

Validation rules and common integration errors

The following checks should be performed during mapping and acceptance testing.

Minimum validation rules

• Required identifiers are present and non-empty.
• planned_end is later than planned_start.
• end_time is later than start_time.
• Durations and quantities are not negative.
• resource_code resolves to a known machine/resource mapping.
• BDE/MES order_id values match ERP planning order_id values exactly.
• Production-time unit and basis are known.
• Timezone interpretation is explicit.
• Unknown optional values are not silently replaced with meaningful-looking zeros.

Common integration errors

Customer handover package

Use this section as the practical integration-kickoff package. Completing these items before implementation reduces mapping iterations and prevents ambiguous assumptions.

Files and documentation to send

• ERP planning sample file or schema containing representative production orders and operations.
• Field dictionary from the customer source system, including descriptions and units.
• Machine/resource master extract showing resource codes and names.
• Plant/site timezone and local timestamp conventions.
• Explanation of setup-time and production-time units and production-time basis.
• Priority semantics if priority is provided.
• BDE/MES interval sample and state/reason code list, if existing interval integration is in scope.
• Target interface specification for Novo AI machine-state write-back, if write-back is in scope.
• Network/security requirements and environment information.
• Named technical owner and business/production-planning owner for validation.

Integration worksheet

Recommended implementation sequence

Final go-live readiness check

• All required direct-integration fields are available or an agreed alternative mapping exists.
• Stable order, product, operation, and resource identifiers have been validated.
• Timestamp timezone behavior has been tested, including local-time edge cases where relevant.
• Quantity units and all time units are documented.
• Production-time basis is explicitly known.
• Resource mapping has been validated against actual machines.
• ERP update/replanning behavior has been tested.
• BDE/MES state mapping has been approved if included.
• Write-back target and permissions have been tested if included.
• Business users have validated representative records end-to-end.
• Technical owners know how schema/code-list changes will be communicated after go-live.