Mapping Engine

A mapping converts a record of one model into a record of another model. You write mappings in a Portable Data Module (PDM).

A PDM has two kinds of mapping:

• A mapping block. This block holds declarative rules. One block can convert data in both directions.
• A mapFn function. This function holds JavaScript. It converts data in one direction only.

For complete modules with sample input and output, see Mapping Examples.

Where Precept runs a mapping

Precept runs mappings in two places. Each place supports a different set of rules.

Feature	API actions	Data ingestion
How Precept runs the rules	In memory, one record at a time	Compiled to one SQL SELECT
mapping block	Yes	Yes
mapFn	Yes	No
<-> between top-level fields	Yes	Yes
<-> with a dotted path or a field list	Yes	No
match blocks	Yes	No
separator, coalesce, template	Yes	No
slice	Yes	Yes

API actions

To get a mapped result from an action, add the precept-mapping-target parameter to the Accept header. The value is the full name of the target model, in the form vendor/module.Model.

GET /rest/acme/crm/resource/Contact/list
Accept: application/json; precept-mapping-target="acme/people.Person"

Precept then looks for a mapping block between the model of the action and the target model. If a mapping block exists, Precept uses it. If no mapping block exists, Precept uses a mapFn for the same pair of models. If neither exists, Precept returns the data without changes.

A mapping block converts one record. If the action returns a JSON array, the mapped result is an empty object. For an action that returns an array, use a mapFn. A mapFn receives the full result, so it can map each item.

Data ingestion

When a data ingestion read has a target model, Precept compiles the mapping block to SQL. Only <-> rules between top-level fields and slice rules become SQL columns. Precept does not apply other rules in this path. Each field of the target model that no rule fills becomes NULL.

The mapping block

A mapping block names two models. The model before <-> is the left model. The model after <-> is the right model.

vendor acme
name people

model Contact {
  first_name: string
  last_name: string
}

model Person {
  firstName: string
  lastName: string
}

mapping Contact <-> Person {
  first_name <-> firstName
  last_name <-> lastName
}

Each line in the block is one rule. The left side of a rule names a field of the left model. The right side names a field of the right model. The order of the models is not a direction. Precept uses the same block when it reads Contact data and when it reads Person data.

Rule forms

Form	Meaning
a <-> b	Copy a to b, and b to a.
a.x <-> b	Use a nested field on the left side. Precept creates the nested object when it writes.
a <-> {x, y}	Nest the fields x and y of the right model into the object a, and flatten them back.
a? <-> b	If the source value is null or missing, do not write the target field.
a <- coalesce(x, y, "default")	Write the first value that is not empty into a. See coalesce.
a <- match { ... } -> b	Translate values. See match blocks.
a <- separator(" ") | {x, y} -> b	Join fields into one string, or split one string into fields. See separator.
a <- template(...) -> b	Parse a structured string into parts, or build the string from parts. See template.
a | slice(1, 9) <-> b	Read a fixed-width part of a string. See slice.

A field list is a list of field names in braces, for example {first, last}. Precept matches the items of a field list to values by position.

match blocks

A match block translates one set of values into another set of values. Each line is a pair. The value on the left belongs to the left model. The value on the right belongs to the right model.

vendor acme
name status

model Contact {
  status: string
}

model Person {
  is_active: boolean
}

mapping Contact <-> Person {
  status <- match {
    "Active" <-> true
    "Inactive" <-> false
    _ <-> null
  } -> is_active
}

Value	Meaning
"text"	A string
true, false	A boolean
null	No value
_	The default. Precept uses the other side of this pair when no other pair matches.

When Precept reads Contact data, "Active" becomes true and "Pending" becomes null. When Precept reads Person data, true becomes "Active".

If no pair matches and the block has no _ pair, Precept does not write the target field.

Function reference

Precept has seven mapping functions: separator, coalesce, template, slice, head, tail and sort_by. This list is complete. It is the functionRegistry in packages/pdm-grammar/src/functions.mts.

If a rule calls another name, the PDM parses, but the mapping fails at runtime with the error Function <name> is not registered.

Six functions are reversible: they work in two directions. The table below shows what each direction does.

Function	When Precept reads the model that has the single value	When Precept reads the model that has the parts
separator(sep)	Splits the string into parts	Joins the parts into one string
coalesce(f1, f2, ..., "default")	Writes the value to one field (see below)	Selects the first value that is not empty
template(pattern)	Extracts the named parts from the string	Builds the string from the named parts
slice(start, length)	Extracts one fixed-width part	Builds a fixed-width string from the parts

Function	When Precept reads the model that has the list	When Precept reads the model that has the single value
head(mode)	Selects the first item	Writes the value as a list of one item
tail(mode)	Selects the last item	Writes the value as a list of one item

sort_by(key) is one-way. It runs in one direction only (see sort_by).

separator

separator(sep)

separator joins values into one string with sep between them. In the other direction, it splits a string at each sep. Use separator with a field list.

vendor acme
name names

model Contact {
  fullName: string
}

model Name {
  first: string
  last: string
}

model Person {
  name: Name
}

mapping Contact <-> Person {
  fullName <- separator(" ") | {first, last} -> name
}

When Precept reads Contact data, "Jane Smith" becomes { "name": { "first": "Jane", "last": "Smith" } }. When Precept reads Person data, the two names become "Jane Smith".

If you leave out -> name, Precept reads and writes the fields first and last at the top level of the right model.

When Precept joins, it skips values that are null. When Precept splits, it fills the fields by position. If the string has fewer parts than the field list, the remaining fields are null. A ? on a field does not change the positions.

coalesce

coalesce(field1, field2, ..., "default")

coalesce writes the first value that is not empty. A value is empty when it is null, missing or "". If all values are empty, coalesce writes the default. The default is optional.

vendor acme
name display

model Contact {
  displayName: string
}

model Person {
  fullName: string
  email: string
}

mapping Contact <-> Person {
  displayName <- coalesce(fullName, email, "Unknown")
}

When Precept reads Person data, displayName is fullName. If fullName is empty, displayName is email. If both are empty, displayName is "Unknown".

The other direction uses a fixed rule. If the value contains @, Precept writes it to the second field. If the value is not empty and is not the default, Precept writes it to the first field. Otherwise, Precept writes no field.

template

template(`pattern`)

template parses a structured string into named parts. In the other direction, it builds the string from the parts. Write the pattern in backticks.

vendor acme
name phones

model PhoneParts {
  countryCode: string
  areaCode: string
  localNumber: string
}

model Contact {
  parts: PhoneParts
}

model PhoneRecord {
  phone: string
}

mapping Contact <-> PhoneRecord {
  parts <- template(`{countryCode:num(1-3)}-{areaCode:num(3)}-{localNumber:num(4)}`) -> phone
}

Put the raw string on the right side of the rule, and the object with the parts on the left side. When Precept reads PhoneRecord data, "44-555-1234" becomes { "parts": { "countryCode": "44", "areaCode": "555", "localNumber": "1234" } }. When Precept reads Contact data, the parts become "44-555-1234".

If the string does not match the pattern, the result is an empty object.

A pattern is text with segments in braces. Each segment has the form {name:type}. Text outside the braces must match exactly.

Type	Matches
alpha	One or more letters
alpha(N)	Exactly N letters
alpha(N-M)	N to M letters
num	One or more digits
num(N), num(N-M)	Exactly N digits, or N to M digits
num.num	A decimal number
alnum, alnum(N)	Letters and digits
word, word(N)	Letters, digits and underscores
ws	One or more whitespace characters
ws?	Zero or more whitespace characters
any	One or more characters of any kind
any?	Zero or more characters of any kind
N	Exactly N characters of any kind
[ABC]	One character from the set
"text"	The exact text

Two special segment forms are available:

• {_:type} matches the text and discards it.
• {name?:type} makes the segment optional.

slice

slice(start, length)

slice reads a fixed-width part of a string. start is the position of the first character. The first character of the string is position 1. length is the number of characters. Precept removes spaces from the start and end of the result.

In the other direction, slice pads the value with spaces to length characters. Then Precept puts it at position start. If several slice rules write to the same field, Precept combines them into one fixed-width string.

slice is the only function that works in data ingestion.

vendor acme
name positions

model Parsed {
  cusip: string
  code: string
  amount: number
}

model RawFile {
  packed_data: string
  amount: number
}

mapping Parsed <-> RawFile {
  packed_data | slice(1, 9) <-> cusip
  packed_data | slice(10, 5) <-> code
  amount <-> amount
}

When Precept reads RawFile data, "IBM123456ABCDE" becomes cusip "IBM123456" and code "ABCDE". In data ingestion, the same rules compile to TRIM(SUBSTR("packed_data", 1, 9)) AS "cusip".

Model order for slice

Put the model with the packed string on the right side of the mapping header. Then put the packed field on the left side of each slice rule, as in the example above. If you put the packed model on the left side of the header, the slice rules fail. In data ingestion, the target fields are NULL. In an API action, the mapping stops with an error.

head and tail

head(mode)
tail(mode)

head reads the first item of a list. tail reads the last item. If the list is empty or null, the result is null.

In the other direction, Precept writes the value as a list that contains only that value. Precept does not read the list that the target already has, so the other items of that list are not kept.

mode tells Precept what to write when the value is null. It is optional.

mode	Precept writes
"clear" (default)	An empty list: []
"pop"	An empty list: []
"nullify"	A list with one null item: [null]

Any other mode causes the error head("<mode>"): the null mode must be "clear", "pop" or "nullify".

vendor acme
name tickets

model Old {
  assignee: string
}

model New {
  assignees: string[]
}

mapping Old <-> New {
  assignee | head("nullify") <-> assignees
}

When Precept reads New data, {"assignees": ["Alice", "Bob"]} becomes {"assignee": "Alice"}. When Precept reads Old data, {"assignee": "Carol"} becomes {"assignees": ["Carol"]}, and {"assignee": null} becomes {"assignees": [null]}.

To read an item of a nested list, chain the functions: cell | head | tail <-> grid reads the last item of the first inner list. See Mapping Examples.

sort_by

sort_by(key)
sort_by(key, descending)

sort_by sorts a list of records by the field key. The order is ascending. Add descending for descending order. Records with the same key stay in their original order. Records with a null key go last in both orders.

sort_by is one-way. It runs only when Precept writes the field that its pipeline is attached to. In the other direction, Precept copies the list without changes.

vendor acme
name queue

model Assignee {
  name: string
  priority: number
}

model Ranked {
  ranked: Assignee[]
}

model Queue {
  queue: Assignee[]
}

mapping Ranked <-> Queue {
  ranked | sort_by(priority, descending) <-> queue
}

When Precept reads Queue data, this list:

{"queue": [{"name": "A", "priority": 1}, {"name": "B", "priority": 2}, {"name": "C", "priority": null}]}

becomes:

{"ranked": [{"name": "B", "priority": 2}, {"name": "A", "priority": 1}, {"name": "C", "priority": null}]}

When Precept reads Ranked data, Precept copies ranked to queue without sorting. For more examples, see Mapping Examples.

The mapFn function

A mapFn is a JavaScript function that converts a record of one model into a record of another model.

vendor acme
name users

model User {
  id: string
  name: string
  email: string
}

model Person {
  firstName: string
  lastName: string
  email: string
}

mapFn (user: User): Person {=
  const [firstName, ...rest] = user.name.split(' ')

  return {
    firstName: firstName || '',
    lastName: rest.join(' '),
    email: user.email
  }
=}

• The parameter in parentheses names the source record and its model.
• The model after : is the target model.
• The body is between {= and =}. The body must return the target record.
• A mapFn converts data in one direction only. For the other direction, write a second mapFn.
• You can give a mapFn a name, for example mapFn toPerson (user: User): Person. Precept does not use the name.
• Precept uses one mapFn for each pair of models. If a module has two mapFn functions for the same pair, Precept uses the last one.
• If a mapping block exists for the same pair of models, Precept uses the mapping block and not the mapFn.