Transfer data type system
The Data type system provides classes and conversion rules representing common data types. It allows for constructing in-memory representations of types, inspecting said representations, combining them to derive new types, and serializing and deserializing them.
Type system provides built-in types similar to the most commonly used types. For convenience, the types are divided into the following groups:
Booleanis a boolean type with two possible values:
Int8is a signed integer,
Int16is a signed integer,
Int32is a signed integer,
Int64is a signed integer,
Uint8is an unsigned integer,
Uint16is an unsigned integer,
Uint32is an unsigned integer,
Uint64is an unsigned integer,
Floating point numbers
Floating point numbers are number types that store values in the form of a mantissa and exponent, in accordance with IEEE-754 .
floatis a floating point number, four bytes.
doubleis a decimal number; for type fullness, Transfer stores it in a string format. It can cast it to the most precise floating point data type in most databases.
Strings are types for working with text and binary data.
stringis an arbitrary binary blob.
utf8is text encoded in Utf8 .
Time types are used for storing time. This type represents an instant in time with nanosecond precision, but for some defined types, in can round down to:
date- a day.
datetime- a second.
timestamp- the most precise time representation.
Additional Data Types
Transfer employs an open-type data system.
Open-type systems declare an infinite type conversion by adding opaque data type. In a matter of simplicity, we define everything that's not as simple as the type above as ANY data type. The only requirement for this type is to be somehow serializable (usually JSON).
Each such type is open for improvement by adding extra type tags. This tag is usually invisible to the user and only used by the system itself (or as part of target data serialization). Most ANY types are, in fact, container data types.
Container data types
Container types are parametric types that provide common data structures. We define several containers to simplify data integration.
optional <T>- a container that may or may not contain a value of type
T. Most databases have such a concept as nullable columns. Each nullable column transforms into this container type.
list <T>- a list of values of type
T. In general, a List is translated into a dynamic array. Most databases support this container.
dict <K, V>- an associative container that maps to one key of type
Kexactly one value of type
If we face a source or target with limited complex type support, we downcast it to JSON built-in.