DLR C API¶

enum DLRModelElemType¶

Values:

enumerator HEXAGON_LIB¶

enumerator NEO_METADATA¶

enumerator TVM_GRAPH¶

enumerator TVM_LIB¶

enumerator TVM_PARAMS¶

enumerator RELAY_EXEC¶

enumerator HEXAGON_LIB

enumerator NEO_METADATA

enumerator TVM_GRAPH

enumerator TVM_LIB

enumerator TVM_PARAMS

enumerator RELAY_EXEC

typedef void *DLRModelHandle¶: Handle for DLRModel.

typedef void *(*DLRMallocFunctionPtr)(size_t)¶: A pointer to a malloc-like function.

typedef void (*DLRFreeFunctionPtr)(void*)¶: A pointer to a free-like function.

typedef void *(*DLRMemalignFunctionPtr)(size_t, size_t)¶: A pointer to a memalign-like function.

typedef struct ModelElem DLRModelElem¶

int CreateDLRModel(DLRModelHandle *handle, const char *model_path, int dev_type, int dev_id)¶

Creates a DLR model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The pointer to save the model handle.
model_path: Path to the folder containing the model files, or colon-separated list of folders containing model files, or colon-separated list of paths to model files
dev_type: Device type. Valid values are in the DLDeviceType enum in dlpack.h.
dev_id: Device ID.

int CreateDLRModelFromModelElem(DLRModelHandle *handle, const DLRModelElem *model_elems, size_t model_elems_size, int dev_type, int dev_id)¶

Creates a DLR model from model elements.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The pointer to save the model handle.
model_elems: DLR Model elements. Element can be file path or data pointer in memory.
dev_type: Device type. Valid values are in the DLDeviceType enum in dlpack.h.
dev_id: Device ID.

int CreateDLRPipeline(DLRModelHandle *handle, int num_models, const char **model_paths, int dev_type, int dev_id)¶

Creates a DLR pipeline model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The pointer to save the model handle.
num_models: Number of items in model_paths array
model_paths: Paths to the folders containing the models files, or colon-separated list of folders (or files) if model files stored in different locations
dev_type: Device type. Valid values are in the DLDeviceType enum in dlpack.h.
dev_id: Device ID.

int DeleteDLRModel(DLRModelHandle *handle)¶

Deletes a DLR model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().

int RunDLRModel(DLRModelHandle *handle)¶

Runs a DLR model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().

int GetDLRNumInputs(DLRModelHandle *handle, int *num_inputs)¶

Gets the number of inputs.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
num_inputs: The pointer to save the number of inputs.

int GetDLRNumWeights(DLRModelHandle *handle, int *num_weights)¶

Gets the number of weights.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
num_weights: The pointer to save the number of weights.

int GetDLRInputName(DLRModelHandle *handle, int index, const char **input_name)¶

Gets the name of the index-th input.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index of the input.
input_name: The pointer to save the name of the index-th input.

int GetDLRInputType(DLRModelHandle *handle, int index, const char **input_type)¶

Gets the type of the index-th input.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index of the input.
input_type: The pointer to save the type of the index-th input.

int GetDLRWeightName(DLRModelHandle *handle, int index, const char **weight_name)¶

Gets the name of the index-th weight.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index of the weight.
input_name: The pointer to save the name of the index-th weight.

int SetDLRInput(DLRModelHandle *handle, const char *name, const int64_t *shape, const void *input, int dim)¶

Sets the input according the node name.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The input node name.
shape: The input node shape as an array.
input: The data for the input as an array.
dim: The dimension of the input data.

int SetDLRInputTensor(DLRModelHandle *handle, const char *name, void *tensor)¶

Sets the input according the node name from existing DLTensor. Can only be used with TVM models (GraphRuntime and VMRuntime)

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The input node name.
tensor: The input DLTensor.

int SetDLRInputTensorZeroCopy(DLRModelHandle *handle, const char *name, void *tensor)¶

Sets the input from existing DLTensor without copying data. Can only be used with TVM models (GraphRuntime). Input tensor device must match the device of the model, and data must be alligned to 128 bytes. GetDLRInput cannot be used for inputs set via SetDLRInputZeroCopy.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The input node name.
tensor: The input DLTensor.

int GetDLRInput(DLRModelHandle *handle, const char *name, void *input)¶

Gets the current value of the input according the node name.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The input node name.
input: The current value of the input will be copied to this buffer.

int GetDLRInputShape(DLRModelHandle *handle, int index, int64_t *shape)¶

Gets the shape of the index-th input.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th input.
shape: The pointer to save the shape of index-th input. This should be a pointer to an array of size “dim” from GetDLRInputSizeDim().

int GetDLRInputSizeDim(DLRModelHandle *handle, int index, int64_t *size, int *dim)¶

Gets the size and dimension of an input.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th input.
size: The pointer to save the size of the index-th input.
dim: The pointer to save the dimension of the index-th output.

int GetDLROutputShape(DLRModelHandle *handle, int index, int64_t *shape)¶

Gets the shape of the index-th output.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
shape: The pointer to save the shape of index-th output. This should be a pointer to an array of size “dim” from GetDLROutputSizeDim().

int GetDLROutput(DLRModelHandle *handle, int index, void *out)¶

Gets the index-th output from the model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
out: The pointer to save the output data. This should be a pointer to an array of size “size” from GetDLROutputSizeDim().

int GetDLROutputPtr(DLRModelHandle *handle, int index, const void **out)¶

Gets the index-th output from the model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
out: Storage to save output pointer

int GetDLROutputTensor(DLRModelHandle *handle, int index, void *tensor)¶

Gets the index-th output from the model and copies it into the given DLTensor. Can only be used with TVM models (GraphRuntime and VMRuntime)

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
tensor: The pointer to an existing/allocated DLTensor to copy the output into.

int GetDLROutputManagedTensorPtr(DLRModelHandle *handle, int index, const void **tensor)¶

Gets the index-th output from the model and sets the pointer to it. Can only be used with TVM models (GraphRuntime and VMRuntime)

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
tensor: The pointer to an unallocated DLManagedTensor pointer, will be set by this function to point to an internal DLManagedTensor.

int GetDLRNumOutputs(DLRModelHandle *handle, int *num_outputs)¶

Gets the number of outputs.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
num_outputs: The pointer to save the number of outputs.

int GetDLROutputSizeDim(DLRModelHandle *handle, int index, int64_t *size, int *dim)¶

Gets the size and dimension of an output.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index-th output.
size: The pointer to save the size of the index-th output.
dim: The pointer to save the dimension of the index-th output.

int GetDLROutputType(DLRModelHandle *handle, int index, const char **output_type)¶

Gets the type of the index-th output.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
index: The index of the output.
output_type: The pointer to save the type of the index-th output.

int GetDLRHasMetadata(DLRModelHandle *handle, bool *has_metadata)¶

Check if metadata file is found in the compilation artifact.

Parameters

handle: The model handle returned from CreateDLRModel().
has_metadata: The pointer to save boolean value to indicate the presence of metadata file.

int GetDLROutputName(DLRModelHandle *handle, const int index, const char **name)¶

Gets the output node names of the uncompiled model from the metadata file.

Parameters

handle: The model handle returned from CreateDLRModel().
names: The pointer to save array containing output node names.

int GetDLROutputIndex(DLRModelHandle *handle, const char *name, int *index)¶

Gets the output node index using the node name.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The pointer pointing to the output node name.
index: The pointer to save the corresponding index of the output node.

int GetDLROutputByName(DLRModelHandle *handle, const char *name, void *out)¶

Gets the output of the node of the given name from the model.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The name of the output node.
out: The pointer to save the output data. This should be a pointer to an array of size “size” from GetDLROutputSizeDim().

const char *DLRGetLastError()¶

Gets the last error message.

Return: Null-terminated string containing the error message.

int GetDLRBackend(DLRModelHandle *handle, const char **name)¶

Gets the name of the backend (“tvm”, “treelite” or “tflite”)

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
name: The pointer to save the null-terminated string containing the name.

int GetDLRDeviceType(const char *model_path)¶

Gets the DLDeviceType (DLDeviceType::kDLCPU, DLDeviceType::kDLGPU, etc)

Return

DLDeviceType enum for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

model_path: Path to the folder containing the model files, or colon-separated list of folders (or files) if model files

int GetDLRVersion(const char **out)¶

Get DLR version.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

out: The pointer to save the null-terminated string containing the version.

int SetDLRNumThreads(DLRModelHandle *handle, int threads)¶

Set the number of threads available to DLR.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
threads: number of threads

int UseDLRCPUAffinity(DLRModelHandle *handle, int use)¶

Enable or disable CPU Affinity.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

handle: The model handle returned from CreateDLRModel().
use: 0 to disable, 1 to enable

int SetDLRCustomAllocatorMalloc(DLRMallocFunctionPtr custom_malloc_fn)¶

Set custom allocator malloc function. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorFree and SetDLRCustomAllocatorMemalign.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

custom_memalign_fn: Function pointer to memalign-like function.

int SetDLRCustomAllocatorFree(DLRFreeFunctionPtr custom_free_fn)¶

Set custom allocator free function. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorMalloc and SetDLRCustomAllocatorMemalign.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

custom_free_fn: Function pointer to free-like function.

int SetDLRCustomAllocatorMemalign(DLRMemalignFunctionPtr custom_memalign_fn)¶

Set custom allocator memalign function. memalign is used heavily by the TVM and RelayVM backends. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorMalloc and SetDLRCustomAllocatorFree.

Return

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

Parameters

custom_memalign_fn: Function pointer to memalign-like function.

DLR_ALLOC_TYPEDEF¶

DLR_MODEL_ELEM¶

struct ModelElem¶

Table Of Contents

DLR C API¶