HLS合成为一整个视频之解决DTS不连续问题

马上就说

发布于 2021-04-13 15:15:58

3K0

发布于 2021-04-13 15:15:58

文章被收录于专栏：码上就说

HLS合成为一整个视频拓展篇一文中讲述了HLS合成为一个MP4视频时出现了时间戳不连续导致的合成失败的问题.

Application provided invalid, non monotonically increasing dts to muxer in stream 1: 11264 >= 0

这个问题已经找到解决的方案了.

首先还是要回到源码, ffmpeg源码中的mux.c文件======>compute_muxer_pkt_fields函数

我截取了最核心的判断逻辑

    if (st->cur_dts && st->cur_dts != AV_NOPTS_VALUE &&
        ((!(s->oformat->flags & AVFMT_TS_NONSTRICT) &&
          st->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE &&
          st->codecpar->codec_type != AVMEDIA_TYPE_DATA &&
          st->cur_dts >= pkt->dts) || st->cur_dts > pkt->dts)) {
        av_log(s, AV_LOG_ERROR,
               "Application provided invalid, non monotonically increasing dts to muxer in stream %d: %s >= %s\n",
               st->index, av_ts2str(st->cur_dts), av_ts2str(pkt->dts));
        return AVERROR(EINVAL);
    }

现在我们一个一个拆解一下判断逻辑:

（1）条件一: st->cur_dts 说明当前的dts要存在,而且要大于0
（2）条件二: st->cur_dts != AV_NOPTS_VALUE 说明当前dts一定不能未知, 不能等于AV_NOPTS_VALUE #define AV_NOPTS_VALUE ((int64_t)UINT64_C(0x8000000000000000))
（3）条件三: ((!(s->oformat->flags & AVFMT_TS_NONSTRICT) && st->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE && st->codecpar->codec_type != AVMEDIA_TYPE_DATA && st->cur_dts >= pkt->dts) 条件三比较复杂, 并且都是&&, 说明只要有一个条件不成立, 整个条件都不成立.
（4）条件四: st->cur_dts > pkt->dts 当前的dts一定要大于avpacket中的dts, 这样的条件未免有点苛刻了.

条件三是有必要拿出来讲讲的,

!(s->oformat->flags & AVFMT_TS_NONSTRICT)

s->oformat->flags是一个枚举的标记, 表示当前重新封装的视频遵守什么规则

/// Demuxer will use avio_open, no opened file should be provided by the caller.
#define AVFMT_NOFILE        0x0001
#define AVFMT_NEEDNUMBER    0x0002 /**< Needs '%d' in filename. */
#define AVFMT_SHOW_IDS      0x0008 /**< Show format stream IDs numbers. */
#define AVFMT_GLOBALHEADER  0x0040 /**< Format wants global header. */
#define AVFMT_NOTIMESTAMPS  0x0080 /**< Format does not need / have any timestamps. */
#define AVFMT_GENERIC_INDEX 0x0100 /**< Use generic index building code. */
#define AVFMT_TS_DISCONT    0x0200 /**< Format allows timestamp discontinuities. Note, muxers always require valid (monotone) timestamps */
#define AVFMT_VARIABLE_FPS  0x0400 /**< Format allows variable fps. */
#define AVFMT_NODIMENSIONS  0x0800 /**< Format does not need width/height */
#define AVFMT_NOSTREAMS     0x1000 /**< Format does not require any streams */
#define AVFMT_NOBINSEARCH   0x2000 /**< Format does not allow to fall back on binary search via read_timestamp */
#define AVFMT_NOGENSEARCH   0x4000 /**< Format does not allow to fall back on generic search */
#define AVFMT_NO_BYTE_SEEK  0x8000 /**< Format does not allow seeking by bytes */
#define AVFMT_ALLOW_FLUSH  0x10000 /**< Format allows flushing. If not set, the muxer will not receive a NULL packet in the write_packet function. */
#define AVFMT_TS_NONSTRICT 0x20000 /**< Format does not require strictly
                                        increasing timestamps, but they must
                                        still be monotonic */
#define AVFMT_TS_NEGATIVE  0x40000 /**< Format allows muxing negative
                                        timestamps. If not set the timestamp
                                        will be shifted in av_write_frame and
                                        av_interleaved_write_frame so they
                                        start from 0.
                                        The user or muxer can override this through
                                        AVFormatContext.avoid_negative_ts
                                        */

#define AVFMT_SEEK_TO_PTS   0x4000000 /**< Seeking is based on PTS */

s->oformat->flags & AVFMT_TS_NONSTRICT 表示输出的视频format格式已经设置了非严格递增的时间戳, 这一点很重要, 我们在HLS合成MP4之前, 需要设置这一参数, 然后s->oformat->flags & AVFMT_TS_NONSTRICT 结果为true, !(s->oformat->flags & AVFMT_TS_NONSTRICT)为false, 条件三自然就不成立.

条件四的判断我觉得有点鸡肋, 因为已经不要dts是严格递增了, 为什么还要加这个判断了, 我的策略是直接去掉这个判断. 所以我将ffmpeg中源码的最终判断条件该成如下:

    if (st->cur_dts && st->cur_dts != AV_NOPTS_VALUE &&
        ((!(s->oformat->flags & AVFMT_TS_NONSTRICT) &&
          st->codecpar->codec_type != AVMEDIA_TYPE_SUBTITLE &&
          st->codecpar->codec_type != AVMEDIA_TYPE_DATA &&
          st->cur_dts >= pkt->dts))) {
        av_log(s, AV_LOG_ERROR,
               "Application provided invalid, non monotonically increasing dts to muxer in stream %d: %s >= %s\n",
               st->index, av_ts2str(st->cur_dts), av_ts2str(pkt->dts));
        return AVERROR(EINVAL);
    }

而且我在demo中还加上dts和pts的严格校验代码, 核心思想就是保证pts或者dts不能出现AV_NOPTS_VALUE

        if (pkt.pts == AV_NOPTS_VALUE) {
            if (pkt.dts != AV_NOPTS_VALUE) {
                pkt.pts = pkt.dts;
                last_dts = pkt.dts;
            } else {
                pkt.pts = last_dts + 1;
                pkt.dts = pkt.pts;
                last_dts = pkt.pts;
            }
        } else {
            if (pkt.dts != AV_NOPTS_VALUE) {
                last_dts = pkt.dts;
            } else {
                pkt.dts = pkt.pts;
                last_dts = pkt.dts;
            }
        }

        if (pkt.pts < pkt.dts) {
            pkt.pts = pkt.dts;
        }

执行测试用例，源视频如下：

PD1824:/sdcard/下载/.video-cache/481b863cc1d8ac51b3c3dd1a6f82cc3d # ls
481b863cc1d8ac51b3c3dd1a6f82cc3d.m3u8 video_1.ts  video_14.ts video_19.ts video_23.ts video_28.ts video_32.ts video_37.ts video_41.ts video_46.ts video_50.ts video_9.ts 
remote.m3u8                           video_10.ts video_15.ts video_2.ts  video_24.ts video_29.ts video_33.ts video_38.ts video_42.ts video_47.ts video_51.ts 
ts_video.info                         video_11.ts video_16.ts video_20.ts video_25.ts video_3.ts  video_34.ts video_39.ts video_43.ts video_48.ts video_6.ts  
video.info                            video_12.ts video_17.ts video_21.ts video_26.ts video_30.ts video_35.ts video_4.ts  video_44.ts video_49.ts video_7.ts  
video_0.ts                            video_13.ts video_18.ts video_22.ts video_27.ts video_31.ts video_36.ts video_40.ts video_45.ts video_5.ts  video_8.ts

执行后得到test.mp4:

PD1824:/sdcard/下载/.video-cache/481b863cc1d8ac51b3c3dd1a6f82cc3d # ls -hl test.mp4                                                                                                                          
-rw-rw---- 1 root sdcard_rw 40M 2021-03-29 15:35 test.mp4

具体见:

https://github.com/JeffMony/JeffFFmpegDemo

https://github.com/JeffMony/VideoDownloader