Table Of Contents

  1. Docs
  2. DLR C API

DLR C API

enum DLRModelElemType

Values:

enumerator HEXAGON_LIB
enumerator NEO_METADATA
enumerator TVM_GRAPH
enumerator TVM_LIB
enumerator TVM_PARAMS
enumerator RELAY_EXEC
enumerator HEXAGON_LIB
enumerator NEO_METADATA
enumerator TVM_GRAPH
enumerator TVM_LIB
enumerator TVM_PARAMS
enumerator RELAY_EXEC
typedef void *DLRModelHandle

Handle for DLRModel.

typedef void *(*DLRMallocFunctionPtr)(size_t)

A pointer to a malloc-like function.

typedef void (*DLRFreeFunctionPtr)(void*)

A pointer to a free-like function.

typedef void *(*DLRMemalignFunctionPtr)(size_t, size_t)

A pointer to a memalign-like function.

typedef struct ModelElem DLRModelElem
int CreateDLRModel(DLRModelHandle *handle, const char *model_path, int dev_type, int dev_id)

Creates a DLR model.

Creates a DLR model.

Parameters

  • handle – The pointer to save the model handle.

  • model_path – Path to the folder containing the model files, or colon-separated list of folders containing model files, or colon-separated list of paths to model files

  • dev_type – Device type. Valid values are in the DLDeviceType enum in dlpack.h.

  • dev_id – Device ID.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int CreateDLRModelFromModelElem(DLRModelHandle *handle, const DLRModelElem *model_elems, size_t model_elems_size, int dev_type, int dev_id)

Creates a DLR model from model elements.

Parameters

  • handle – The pointer to save the model handle.

  • model_elems – DLR Model elements. Element can be file path or data pointer in memory.

  • dev_type – Device type. Valid values are in the DLDeviceType enum in dlpack.h.

  • dev_id – Device ID.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int CreateDLRPipeline(DLRModelHandle *handle, int num_models, const char **model_paths, int dev_type, int dev_id)

Creates a DLR pipeline model.

Creates a DLR pipeline model.

Parameters

  • handle – The pointer to save the model handle.

  • num_models – Number of items in model_paths array

  • model_paths – Paths to the folders containing the models files, or colon-separated list of folders (or files) if model files stored in different locations

  • dev_type – Device type. Valid values are in the DLDeviceType enum in dlpack.h.

  • dev_id – Device ID.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int DeleteDLRModel(DLRModelHandle *handle)

Deletes a DLR model.

Parameters

handle – The model handle returned from CreateDLRModel().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int RunDLRModel(DLRModelHandle *handle)

Runs a DLR model.

Parameters

handle – The model handle returned from CreateDLRModel().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRNumInputs(DLRModelHandle *handle, int *num_inputs)

Gets the number of inputs.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • num_inputs – The pointer to save the number of inputs.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRNumWeights(DLRModelHandle *handle, int *num_weights)

Gets the number of weights.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • num_weights – The pointer to save the number of weights.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRInputName(DLRModelHandle *handle, int index, const char **input_name)

Gets the name of the index-th input.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index of the input.

  • input_name – The pointer to save the name of the index-th input.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRInputType(DLRModelHandle *handle, int index, const char **input_type)

Gets the type of the index-th input.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index of the input.

  • input_type – The pointer to save the type of the index-th input.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRWeightName(DLRModelHandle *handle, int index, const char **weight_name)

Gets the name of the index-th weight.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index of the weight.

  • input_name – The pointer to save the name of the index-th weight.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRInput(DLRModelHandle *handle, const char *name, const int64_t *shape, const void *input, int dim)

Sets the input according the node name.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The input node name.

  • shape – The input node shape as an array.

  • input – The data for the input as an array.

  • dim – The dimension of the input data.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRInputTensor(DLRModelHandle *handle, const char *name, void *tensor)

Sets the input according the node name from existing DLTensor. Can only be used with TVM models (GraphExecutor and VMRuntime)

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The input node name.

  • tensor – The input DLTensor.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRInputTensorZeroCopy(DLRModelHandle *handle, const char *name, void *tensor)

Sets the input from existing DLTensor without copying data. Can only be used with TVM models (GraphExecutor). Input tensor device must match the device of the model, and data must be alligned to 128 bytes. GetDLRInput cannot be used for inputs set via SetDLRInputZeroCopy.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The input node name.

  • tensor – The input DLTensor.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRInput(DLRModelHandle *handle, const char *name, void *input)

Gets the current value of the input according the node name.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The input node name.

  • input – The current value of the input will be copied to this buffer.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRInputShape(DLRModelHandle *handle, int index, int64_t *shape)

Gets the shape of the index-th input.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th input.

  • shape – The pointer to save the shape of index-th input. This should be a pointer to an array of size “dim” from GetDLRInputSizeDim().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRInputSizeDim(DLRModelHandle *handle, int index, int64_t *size, int *dim)

Gets the size and dimension of an input.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th input.

  • size – The pointer to save the size of the index-th input.

  • dim – The pointer to save the dimension of the index-th output.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputShape(DLRModelHandle *handle, int index, int64_t *shape)

Gets the shape of the index-th output.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • shape – The pointer to save the shape of index-th output. This should be a pointer to an array of size “dim” from GetDLROutputSizeDim().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutput(DLRModelHandle *handle, int index, void *out)

Gets the index-th output from the model.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • out – The pointer to save the output data. This should be a pointer to an array of size “size” from GetDLROutputSizeDim().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputPtr(DLRModelHandle *handle, int index, const void **out)

Gets the index-th output from the model.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • out – Storage to save output pointer

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputTensor(DLRModelHandle *handle, int index, void *tensor)

Gets the index-th output from the model and copies it into the given DLTensor. Can only be used with TVM models (GraphExecutor and VMRuntime)

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • tensor – The pointer to an existing/allocated DLTensor to copy the output into.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputManagedTensorPtr(DLRModelHandle *handle, int index, const void **tensor)

Gets the index-th output from the model and sets the pointer to it. Can only be used with TVM models (GraphExecutor and VMRuntime)

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • tensor – The pointer to an unallocated DLManagedTensor pointer, will be set by this function to point to an internal DLManagedTensor.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRNumOutputs(DLRModelHandle *handle, int *num_outputs)

Gets the number of outputs.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • num_outputs – The pointer to save the number of outputs.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputSizeDim(DLRModelHandle *handle, int index, int64_t *size, int *dim)

Gets the size and dimension of an output.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index-th output.

  • size – The pointer to save the size of the index-th output.

  • dim – The pointer to save the dimension of the index-th output.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLROutputType(DLRModelHandle *handle, int index, const char **output_type)

Gets the type of the index-th output.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • index – The index of the output.

  • output_type – The pointer to save the type of the index-th output.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRHasMetadata(DLRModelHandle *handle, bool *has_metadata)

Check if metadata file is found in the compilation artifact.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • has_metadata – The pointer to save boolean value to indicate the presence of metadata file.

int GetDLROutputName(DLRModelHandle *handle, const int index, const char **name)

Gets the output node names of the uncompiled model from the metadata file.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • names – The pointer to save array containing output node names.

int GetDLROutputIndex(DLRModelHandle *handle, const char *name, int *index)

Gets the output node index using the node name.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The pointer pointing to the output node name.

  • index – The pointer to save the corresponding index of the output node.

int GetDLROutputByName(DLRModelHandle *handle, const char *name, void *out)

Gets the output of the node of the given name from the model.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The name of the output node.

  • out – The pointer to save the output data. This should be a pointer to an array of size “size” from GetDLROutputSizeDim().

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

const char *DLRGetLastError()

Gets the last error message.

Returns

Null-terminated string containing the error message.

int GetDLRBackend(DLRModelHandle *handle, const char **name)

Gets the name of the backend (“tvm”, “treelite” or “tflite”)

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • name – The pointer to save the null-terminated string containing the name.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRDeviceType(const char *model_path)

Gets the DLDeviceType (DLDeviceType::kDLCPU, DLDeviceType::kDLGPU, etc)

Parameters

model_path – Path to the folder containing the model files, or colon-separated list of folders (or files) if model files

Returns

DLDeviceType enum for success, -1 for error. Call DLRGetLastError() to get the error message.

int GetDLRVersion(const char **out)

Get DLR version.

Parameters

out – The pointer to save the null-terminated string containing the version.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRNumThreads(DLRModelHandle *handle, int threads)

Set the number of threads available to DLR.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • threads – number of threads

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int UseDLRCPUAffinity(DLRModelHandle *handle, int use)

Enable or disable CPU Affinity.

Parameters

  • handle – The model handle returned from CreateDLRModel().

  • use – 0 to disable, 1 to enable

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRCustomAllocatorMalloc(DLRMallocFunctionPtr custom_malloc_fn)

Set custom allocator malloc function. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorFree and SetDLRCustomAllocatorMemalign.

Parameters

custom_memalign_fn – Function pointer to memalign-like function.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRCustomAllocatorFree(DLRFreeFunctionPtr custom_free_fn)

Set custom allocator free function. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorMalloc and SetDLRCustomAllocatorMemalign.

Parameters

custom_free_fn – Function pointer to free-like function.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

int SetDLRCustomAllocatorMemalign(DLRMemalignFunctionPtr custom_memalign_fn)

Set custom allocator memalign function. memalign is used heavily by the TVM and RelayVM backends. Must be called before CreateDLRModel or CreateDLRPipeline. It is recommended to use with SetDLRCustomAllocatorMalloc and SetDLRCustomAllocatorFree.

Parameters

custom_memalign_fn – Function pointer to memalign-like function.

Returns

0 for success, -1 for error. Call DLRGetLastError() to get the error message.

DLR_ALLOC_TYPEDEF
DLR_MODEL_ELEM
struct ModelElem
DLR Python API
Additional Options for TensorRT Optimized Models
© Copyright 2019, AWS. Created using Sphinx.