This partially reverts commit 1e9b4c5d84. The make helpers for running the inference code generator were somewhat of a hack due to the desire to integrate a target-specific preprocessor binary as part of the process. Since that is no longer necessary, we can now switch over to just using Bazel for running the code generator.
BUG=b/294230402
The TFLM code_generator takes a TFLite model and generates C/C++ source code that can then be compiled into a binary. This change adds a Bazel macro for invoking the code generator and creating a cc_library with the resulting sources. It also updates the checked-in hello_world example with an appropriate BUILD file and updates the script to use Bazel instead of make.
BUG=b/294230402
The original idea for the codegen preprocessor was to perform init & prepare stages using a simulator and capture the output. This idea is being abandoned in favor of generating the necessary structures from python.
BUG=cleanup
Update the third_party flatbuffers library to v23.5.26, the
current version in upstream TF. Synchronize the override BUILD
and build_defs.bzl files with those from upstream TF at
e4485c98eae.
Also update the Makefile build, which downloads flatbuffers
separately. Rebase the patch applied to the download.
Regenerate the generated-and-checked-in schemas (see
ci/sync_from_upstream_tf.sh and codegen/preprocessor/
update_schema.sh), because they are stamped with the version of
the flatbuffers library, and fail a static_assert if they are
built with a different version of flatbuffers than they were
generated with.
BUG=unsuccessful attempt to fix warning in #2183
The codegen process is a multi-step process that requires compiling,
executing code under simulation, and executing python scripts. To
simplify this workflow, this commit adds Make helper functions for
generating inference source code from a model and creating a binary with
it.
It also updates the hello world example to use these helpers and adds an
update script for keeping the checked in generated source in sync.
BUG=cleanup
The TFLM kernels use the MicroContext and MicroGraph interfaces to fetch
eval tensors and access other parts of the graph. Since codegen does not
have the MicroInterpreter and related objects to serve those, this PR
introduces a new MicroCodegenContext class that serves as both the
MicroContext and MicroGraph.
The MicroCodegenContext is configured with a span of Subgraphs, each of
which includes the inputs, outputs, nodes, tensors and an invocation
function. The code generator will create the data and functions needed
for each Subgraph and initialize the MicroCodegenContext with it. By
having the re-usable MicroCodegenContext, the code generator won't have
to generate nearly as much code.
BUG=b/295174086
The purpose of the preprocessor is to load a model with the TFLM interpreter, allocate tensors, and then capture the resulting data structures. To capture this data, we need to provide structure and serialization to it for ingesting into the code generator. This commit defines a preprocessor data schema with generated C++ and python bindings for it. Unfortunately, we need to check-in the generated code, as the Makefiles are currently not capable of running the flatc compiler. An update script is included to help keep these files in sync.
BUG=b/295076067
The RISC-V toolchain we're using failed to properly discern the TfLiteEvalTensor type for use with an assignment operator. It produced compiler errors for "no match for 'operator=' with operand types TfLiteEvalTensor and a brace-closed initializer list. This PR resolves this issue by explicitly using the TfLiteEvalTensor constructor for the initializer list. We also apply this approach to the TfLiteNode initialization as well, just for consistency.
BUG=#2195
This PR generates all of the TfLiteEvalTensors for the graph. It also generates all of the static buffers used for tensors that were present in the flatbuffer.
BUG=b/295077140
Each operator has its own set of BuiltinOptions, which will translate into an op-specific TfLite Params structure. This change turns Operator into a base class with common functionality, and adds a FullyConnected subclass that knows how to generate the TfLiteFullyConnectedParams struct.
BUG=b/295175961
This PR adds the generation of empty TfLiteContext and TfLiteNode structures, and generates appropriate invocation methods for each subgraph. It also generates the input, output, and intermediate arrays for each TfLiteNode.
BUG=b/295175961
As the next step in the codegen experiment, we want to generate the invoke calls for each layer. This is slightly challenging with the existing sources, as kernels only expose a registration function, not their individual Eval functions. In an effort to keep the code churn to a minimum, this PR introduces an inference only registration structure and function. It includes just two function pointers: invoke and reset. For this CL, we've only introduced it for FullyConnected.
In the code generator, this PR creates a new op_table array in the generated source, with an enum for lookup. It also generates an invoke function for each subgraph, that calls each operator's invoke function.
BUG=295174388
This PR adds a codegen inference example for the hello world model to demonstrate how to invoke the code generator and build the generated source. For now, we're just checking the generated source into the repo to skip over building out the make rules and also ensuring the generated source complies with formatting rules.
This also fixes a minor formatting issue in the source templates, as clang-format now properly complained about it.
BUG=b/295390000
This PR creates the initial code generator scaffolding for performing inference without an interpreter. Currently, this does nothing other create a header and source file from Mako templates. Mako was chosen as a template engine due to existing dependency.
BUG=b/295076487
RJ Ascani