openvela
/
external_ffmpeg
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
118,284 Commits 7 Branches 4 Tags 292 MiB
Commit Graph

255 Commits

Author SHA1 Message Date
openvela-robot 24478030d4 FFMPEG: fix system crash when sim boot whithout camera 
Signed-off-by: lile7 <lile7@xiaomi.com>
 2026-04-20 20:32:11 +08:00
openvela-robot f3b118c812 libavformat/tcp: add local_addr/local_port for network option 
Signed-off-by: jackarain <jack.wgm@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
 2026-04-20 20:32:04 +08:00
openvela-robot d9f83b4739 FFMPEG: add offload option in adevsink. 
Signed-off-by: qiaohaijiao1 <qiaohaijiao1@xiaomi.com>
 2026-04-20 20:32:01 +08:00
openvela-robot 5ab7442253 lavc/codec_desc.c: remove AV_CODEC_PROP_TEXT_SUB property from ARIB_CAPTION 
To support bitmap subtitle output, remove AV_CODEC_PROP_TEXT_SUB
property from codec descriptor for AV_CODEC_ID_ARIB_CAPTION.
This is similar to `libavcodec/libzvbi-teletextdec.c`
(AV_CODEC_ID_DVB_TELETEXT).

Instead, each subtitle decoder has to specify a subtitile format.
`libavcodec/libaribb24.c` uses same AV_CODEC_ID_ARIB_CAPTION and
expects AV_CODEC_PROP_TEXT_SUB to be set, so this adds a line to
specify a format there.

Signed-off-by: rcombs <rcombs@rcombs.me>
 2026-04-20 20:31:57 +08:00
openvela-robot e22343cd0e FFMPEG: add default codec setting in query_format. 
dependson:888283

1, Filters which hook query_formats, set codecs in filter_query_formats.
streamselect is exception because it hook sanitize_formats.

2, Filters which doesn't hook query_formats, set codecs in ff_default_query_formats.

Signed-off-by: qiaohaijiao1 <qiaohaijiao1@xiaomi.com>
 2026-04-20 20:31:54 +08:00
openvela-robot 8161f26c12 avcodec/avutil: move dynamic HDR10+ metadata parsing to libavutil 
Signed-off-by: Raphaël Zumer <rzumer@tebako.net>
Signed-off-by: James Almer <jamrial@gmail.com>
 2026-04-20 20:31:47 +08:00
openvela-robot 96e3f5956c ffmpeg: fix NULL strcmp error 
N/A

Signed-off-by: jihandong <jihandong@xiaomi.com>
 2026-04-20 20:31:43 +08:00
openvela-robot d82f9ae7d8 lavu/frame: improve AVFrame.opaque[_ref] documentation 
Make them match each other, mention interaction with
AV_CODEC_FLAG_COPY_OPAQUE.
 2026-04-20 20:31:34 +08:00
openvela-robot a8e85bba00 ffmpeg: nuttx_dec support temparory pause 
Signed-off-by: jihandong <jihandong@xiaomi.com>
 2026-04-20 20:31:30 +08:00
openvela-robot 51b3e1bb2d avfilter/af_afir: reduce output gain with default parameters 
It was unreasonably high. Also change scaling to reduce
rare quantization errors.
 2026-04-20 20:30:36 +08:00
openvela-robot dcac24dcec ffmpeg: fix uinit error 
0  0x5672bb6e in avformat_free_context (s=0x56d21060) at ffmpeg/libavformat/avformat.c:129
1  0x567ddf8e in adevsink_uninit (ctx=0x56d228b0) at ffmpeg/libavfilter/asink_adevsink.c:169
2  0x56721835 in avfilter_free (filter=0x56d228b0) at ffmpeg/libavfilter/avfilter.c:863
3  0x5672a92e in create_filter (log_ctx=0x56d14590, args=0x56d23050 "format=nuttx:devname=/dev/audio/pcm0p", name=0x56d22f40 "adevsink@pcm0p", index=<optimized out>, ctx=0xfffffffe,
   filt_ctx=0xf6ec3e88) at ffmpeg/libavfilter/graphparser.c:158
4  parse_filter (filt_ctx=filt_ctx@entry=0xf6ec3e88, buf=buf@entry=0xf6ec3e6c, graph=graph@entry=0x56d14590, index=24, log_ctx=0x56d14590) at ffmpeg/libavfilter/graphparser.c:201
5  0x5672af22 in avfilter_graph_parse2 (graph=0x56d14590, filters=<optimized out>, inputs=0xf6ec3ed8, outputs=0xf6ec3edc) at ffmpeg/libavfilter/graphparser.c:438
6  0x5671bc90 in media_graph_load (priv=0x56d16bf0, conf=0x56a5a466 "/etc/media/graph.conf") at media_graph.c:153
7  0x5671c435 in media_graph_create (file=0x56a5a466) at media_graph.c:337
8  0x56713f0b in mediad_main (argc=1, argv=0xf6cb5040) at media_daemon.c:163
9  0x5658de28 in nxtask_startup (entrypt=0x56713e8b <mediad_main>, argc=1, argv=0xf6cb5040) at sched/task_startup.c:70
10 0x5657ff82 in nxtask_start () at task/task_start.c:134
11 0xdeadbeef in ?? ()

Signed-off-by: ligd <liguiding1@xiaomi.com>
 2026-04-20 20:30:32 +08:00
openvela-robot edc934782f avcodec/pnmenc: Check av_image_get_buffer_size() 
Fixes the crash in ticket #10050.
Also ensure that we don't overflow before ff_get_encode_buffer().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
 2026-04-20 20:30:02 +08:00
openvela-robot 1d136bd248 bluelet:fix build error if not enable CONFIG_NET_RPMSG 
Signed-off-by: fangzhenwei <fangzhenwei@xiaomi.com>
 2026-04-20 20:29:57 +08:00
openvela-robot 6d4e16b532 avfilter/af_firequalizer: switch to TX from lavu 2026-04-20 20:29:50 +08:00
openvela-robot 9299ef8b7b ffmpeg/bluelet:fix warning 
libavdevice/bluelet.c:357:33: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘uint32_t’ {aka ‘unsigned int’} [-Wformat=]
  357 |                 "channel_mode=%lu:blocks=%lu:subbands=%lu:alloc_method=%lu:bitpool=%lu",
      |                               ~~^
      |                                 |
      |                                 long unsigned int
      |                               %u
  358 |                 param.channel_mode, param.blocks, param.subbands, param.alloc_method, param.bitpool);
      |                 ~~~~~~~~~~~~~~~~~~
      |                      |
      |                      uint32_t {aka unsigned int}
libavdevice/bluelet.c:357:44: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 5 has type ‘uint32_t’ {aka ‘unsigned int’} [-Wformat=]
  357 |                 "channel_mode=%lu:blocks=%lu:subbands=%lu:alloc_method=%lu:bitpool=%lu",
      |                                          ~~^
      |                                            |
      |                                            long unsigned int
      |                                          %u
  358 |                 param.channel_mode, param.blocks, param.subbands, param.alloc_method, param.bitpool);
      |                                     ~~~~~~~~~~~~
      |                                          |
      |                                          uint32_t {aka unsigned int}

Signed-off-by: fangzhenwei <fangzhenwei@xiaomi.com>
 2026-04-20 20:29:47 +08:00
openvela-robot 0e99873ffa avfilter/af_loudnorm: fix incorrect gain when audio is shorter than 3s 
The input data is multiplied by `s->offset` to get normalized output.
`s->target_tp` and `true_peak` is not in dB,
so `s->offset` should be calculated by division instead of subtraction.

Signed-off-by: Rui Zhu <real.zhurui@gmail.com>
 2026-04-20 20:29:37 +08:00
openvela-robot 5de2d251c8 FFmpeg/bluelet:adapt to 5.1.1 version 
Signed-off-by: jihandong <jihandong@xiaomi.com>
Signed-off-by: fangzhenwei <fangzhenwei@xiaomi.com>
 2026-04-20 20:29:34 +08:00
openvela-robot d4c7951080 avfilter/vf_pseudocolor: add spectral preset 2026-04-20 20:29:22 +08:00
openvela-robot 90eb42bda6 Merge branch 'origin511' into sync511 2026-04-20 20:29:19 +08:00
Limin Wang 25f51f455f avfilter/dnn/dnn_backend_tf: simplify the code with ff_hex_to_data 
please use tools/python/tf_sess_config.py to get the sess_config after that.
note the byte order of session config is in normal order.
bump the MICRO version for the config change.

Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
 2026-04-20 20:19:33 +08:00
Wenlong Ding 9bf86b50ce lavfi/dnn/dnn_backend_native_layer_mathunary: add exp support 
Signed-off-by: Wenlong Ding <wenlong.ding@intel.com>
 2026-04-20 20:18:53 +08:00
Mingyu Yin 01d7cb6160 dnn/native: add native support for dense 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2026-04-20 20:15:25 +08:00
Mingyu Yin e00f9d75a6 dnn_backend_native_layer_mathbinary: add floormod support 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2026-04-20 20:14:42 +08:00
Mingyu Yin b17754d5a4 dnn_backend_native_layer_mathunary: add round support 
Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:32 +08:00
Ting Fu fbdc2f17a0 dnn/native: add native support for avg_pool 
Not support pooling strides in channel dimension yet.

Signed-off-by: Ting Fu <ting.fu@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:30 +08:00
Mingyu Yin 8d7156c45b dnn_backend_native_layer_mathunary: add floor support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import os
import numpy as np
import imageio
from tensorflow.python.framework import graph_util
name = 'floor'

pb_file_path = os.getcwd()
if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)):
    os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))

with tf.Session(graph=tf.Graph()) as sess:
    in_img = imageio.imread('detection.jpg')
    in_img = in_img.astype(np.float32)
    in_data = in_img[np.newaxis, :]
    input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
    y_ = tf.math.floor(input_x*255)/255
    y = tf.identity(y_, name='dnn_out')
    sess.run(tf.global_variables_initializer())
    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])

    with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f:
        f.write(constant_graph.SerializeToString())

    print("model.pb generated, please in ffmpeg path use\n \n \
    python tools/python/convert.py {}_savemodel/model.pb --outdir={}_savemodel/ \n \nto generate model.model\n".format(name,name))

    output = sess.run(y, feed_dict={ input_x: in_data})
    imageio.imsave("out.jpg", np.squeeze(output))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 {}_savemodel/tensorflow_out.md5\n  \
    or\n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow {}_savemodel/out_tensorflow.jpg\n \nto generate output result of tensorflow model\n".format(name, name, name, name))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 {}_savemodel/native_out.md5\n  \
    or \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model={}_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native {}_savemodel/out_native.jpg\n \nto generate output result of native model\n".format(name, name, name, name))

Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
 2026-04-20 20:14:27 +08:00
Mingyu Yin 57e034adad dnn_backend_native_layer_mathunary: add ceil support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import os
import numpy as np
import imageio
from tensorflow.python.framework import graph_util
name = 'ceil'

pb_file_path = os.getcwd()
if not os.path.exists(pb_file_path+'/{}_savemodel/'.format(name)):
    os.mkdir(pb_file_path+'/{}_savemodel/'.format(name))

with tf.Session(graph=tf.Graph()) as sess:
    in_img = imageio.imread('detection.jpg')
    in_img = in_img.astype(np.float32)
    in_data = in_img[np.newaxis, :]
    input_x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
    y = tf.math.ceil( input_x, name='dnn_out')
    sess.run(tf.global_variables_initializer())
    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])

    with tf.gfile.FastGFile(pb_file_path+'/{}_savemodel/model.pb'.format(name), mode='wb') as f:
        f.write(constant_graph.SerializeToString())

    print("model.pb generated, please in ffmpeg path use\n \n \
    python tools/python/convert.py ceil_savemodel/model.pb --outdir=ceil_savemodel/ \n \n \
    to generate model.model\n")

    output = sess.run(y, feed_dict={ input_x: in_data})
    imageio.imsave("out.jpg", np.squeeze(output))

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model=ceil_savemodel/model.pb:input=dnn_in:output=dnn_out:dnn_backend=tensorflow -f framemd5 ceil_savemodel/tensorflow_out.md5\n \n \
    to generate output result of tensorflow model\n")

    print("To verify, please ffmpeg path use\n \n \
    ./ffmpeg -i detection.jpg -vf format=rgb24,dnn_processing=model=ceil_savemodel/model.model:input=dnn_in:output=dnn_out:dnn_backend=native -f framemd5 ceil_savemodel/native_out.md5\n \n \
    to generate output result of native model\n")

Signed-off-by: Mingyu Yin <mingyu.yin@intel.com>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:26 +08:00
Ting Fu 469fc45766 dnn_backend_native_layer_mathunary: add atanh support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')

please uncomment the part you want to test

x_sinh_1 = tf.sinh(x)
x_out = tf.divide(x_sinh_1, 1.176) # sinh(1.0)

x_cosh_1 = tf.cosh(x)
x_out = tf.divide(x_cosh_1, 1.55) # cosh(1.0)

x_tanh_1 = tf.tanh(x)
x__out = tf.divide(x_tanh_1, 0.77) # tanh(1.0)

x_asinh_1 = tf.asinh(x)
x_out = tf.divide(x_asinh_1, 0.89) # asinh(1.0/1.1)

x_acosh_1 = tf.add(x, 1.1)
x_acosh_2 = tf.acosh(x_acosh_1) # accept (1, inf)
x_out = tf.divide(x_acosh_2, 1.4) # acosh(2.1)

x_atanh_1 = tf.divide(x, 1.1)
x_atanh_2 = tf.atanh(x_atanh_1) # accept (-1, 1)
x_out = tf.divide(x_atanh_2, 1.55) # atanhh(1.0/1.1)

y = tf.identity(x_out, name='dnn_out') #please only preserve the x_out you want to test

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu 9bb206f887 dnn_backend_native_layer_mathunary: add acosh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu a2a83a6e47 dnn_backend_native_layer_mathunary: add asinh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu 51eb6e3697 dnn_backend_native_layer_mathunary: add tanh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu 7ba2f132c8 dnn_backend_native_layer_mathunary: add cosh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu ec36314dc0 dnn_backend_native_layer_mathunary: add sinh support 
Signed-off-by: Ting Fu <ting.fu@intel.com>
 2026-04-20 20:14:09 +08:00
Ting Fu ebccdd69a6 dnn_backend_native_layer_mathunary: add atan support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.atan(x)
x2 = tf.divide(x1, 3.1416/4) # pi/4
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:01 +08:00
Ting Fu 7e53492439 dnn_backend_native_layer_mathunary: add acos support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.acos(x)
x2 = tf.divide(x1, 3.1416/2) # pi/2
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:01 +08:00
Ting Fu 146c9191eb dnn_backend_native_layer_mathunary: add asin support 
It can be tested with the model generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.asin(x)
x2 = tf.divide(x1, 3.1416/2) # pi/2
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:14:01 +08:00
Ting Fu 6dfbc96d37 dnn_backend_native_layer_mathunary: add tan support 
It can be tested with the model generated with below python scripy

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 0.78)
x2 = tf.tan(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:13:46 +08:00
Ting Fu 966133bd6a dnn_backend_native_layer_mathunary: add cos support 
It can be tested with the model generated with below python scripy

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 1.5)
x2 = tf.cos(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:13:46 +08:00
Ting Fu 29e32e8e3a dnn_backend_native_layer_mathunary: add sin support 
It can be tested with the model file generated with below python scripy:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.multiply(x, 3.14)
x2 = tf.sin(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo Yejun <yejun.guo@intel.com>
 2026-04-20 20:13:46 +08:00
Ting Fu 2c62ed9150 dnn_backend_native_layer_mathunary: add abs support 
more math unary operations will be added here

It can be tested with the model file generated with below python scripy:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.subtract(x, 0.5)
x2 = tf.abs(x1)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:13:20 +08:00
Guo, Yejun fed73deee2 dnn/native: add native support for minimum 
it can be tested with model file generated with below python script:
import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.minimum(0.7, x)
x2 = tf.maximum(x1, 0.4)
y = tf.identity(x2, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:12:56 +08:00
Guo, Yejun 433d3f3906 dnn/native: add native support for divide 
it can be tested with model file generated with below python script:
import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 2 / x
z2 = 1 / z1
z3 = z2 / 0.25 + 0.3
z4 = z3 - x * 1.5 - 0.3
y = tf.identity(z4, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:12:39 +08:00
Guo, Yejun 1c8338d020 dnn/native: add native support for 'mul' 
it can be tested with model file generated from above python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 0.5 + 0.3 * x
z2 = z1 * 4
z3 = z2 - x - 2.0
y = tf.identity(z3, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:12:39 +08:00
Guo, Yejun 88edd34908 dnn/native: add native support for 'add' 
It can be tested with the model file generated with below python script:

import tensorflow as tf
import numpy as np
import imageio

in_img = imageio.imread('input.jpg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]

x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
z1 = 0.039 + x
z2 = x + 0.042
z3 = z1 + z2
z4 = z3 - 0.381
z5 = z4 - x
y = tf.math.maximum(z5, 0.0, name='dnn_out')

sess=tf.Session()
sess.run(tf.global_variables_initializer())

graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)

print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")

output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:12:39 +08:00
Guo, Yejun c3705ef175 dnn_backend_native_layer_mathbinary: add sub support 
more math binary operations will be added here

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:12:20 +08:00
Guo, Yejun 73a0274ccb convert_from_tensorflow.py: add support when kernel size is 1*1 with one input/output channel (gray image) 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:10:37 +08:00
Guo, Yejun a3d1485050 dnn: add tf.nn.conv2d support for native model 
Unlike other tf.*.conv2d layers, tf.nn.conv2d does not create many
nodes (within a scope) in the graph, it just acts like other layers.
tf.nn.conv2d only creates one node in the graph, and no internal
nodes such as 'kernel' are created.

The format of native model file is also changed, a flag named
has_bias is added, so change the version number.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:10:02 +08:00
Guo, Yejun 7a863118e1 libavfilter/dnn: add layer maximum for native mode. 
The reason to add this layer is that it is used by srcnn in vf_sr.
This layer is currently ignored in native mode. After this patch,
we can add multiple outputs support for native mode.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:09:16 +08:00
Guo, Yejun b6410b7b41 libavfilter/dnn: add header into native model file 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:46 +08:00
Guo, Yejun e79349bb2f dnn: export operand info in python script and load in c code 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:40 +08:00
Guo, Yejun ced0dcbf31 dnn: change .model file format to put layer number at the end of file 
currently, the layer number is at the beginning of the .model file,
so we have to scan twice in python script, the first scan to get the
layer number. Only one scan needed after put the layer number at the
end of .model file.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:40 +08:00
Guo, Yejun dabafcd32b convert_from_tensorflow.py: support conv2d with dilation 
conv2d with dilation > 1 generates tens of nodes in graph, it is not
easy to parse each node one by one, so we do special tricks to parse
the conv2d layer.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:30 +08:00
Guo, Yejun 26dfc526b1 convert_from_tensorflow.py: add option to dump graph for visualization in tensorboard 
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:30 +08:00
Guo, Yejun 0bbbd17a31 dnn: convert tf.pad to native model in python script, and load/execute it in the c code. 
since tf.pad is enabled, the conv2d(valid) changes back to its original behavior.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
 2026-04-20 20:08:18 +08:00
Guo, Yejun 8bcdb534e2 tools/python: add script to convert TensorFlow model (.pb) to native model (.model) 
For example, given TensorFlow model file espcn.pb,
to generate native model file espcn.model, just run:
python convert.py espcn.pb

In current implementation, the native model file is generated for
specific dnn network with hard-code python scripts maintained out of ffmpeg.
For example, srcnn network used by vf_sr is generated with
https://github.com/HighVoltageRocknRoll/sr/blob/master/generate_header_and_model.py#L85

In this patch, the script is designed as a general solution which
converts general TensorFlow model .pb file into .model file. The script
now has some tricky to be compatible with current implemention, will
be refined step by step.

The script is also added into ffmpeg source tree. It is expected there
will be many more patches and community needs the ownership of it.

Another technical direction is to do the conversion in c/c++ code within
ffmpeg source tree. While .pb file is organized with protocol buffers,
it is not easy to do such work with tiny c/c++ code, see more discussion
at http://ffmpeg.org/pipermail/ffmpeg-devel/2019-May/244496.html. So,
choose the python script.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
 2026-04-20 20:07:58 +08:00