Live555 RTSP服务端H264推流笔记（平台RK3399Pro+mpp）

EngineerX_

12721人浏览 · 2022-01-14 14:39:14

EngineerX_ · 2022-01-14 14:39:14 发布

Live555 RTSP服务端H264推流笔记（平台RK3399Pro）

引言
前期准备
- 编译MPP
- 编译live555
部分demo源码记录

引言

因项目要求，需要在RK3399pro上实现RTSP的推流，Live555本身已经实现了本地视频的服务端程序样例，本文主要是基于瑞芯微的MPP编解码+live555实现RTSP的服务端程序，功能基本完成了，怕忘记所以记录一下。

前期准备

编译MPP

首先下载MPP源码（瑞芯微github有最新版）,我为了兼容自己的程序，使用的是SDK中的mpp版本。
进入Mpp源码mpp/build/linux/aarch64中，修改arm.linux.cross.cmake文件中的配置（我没有修改）
运行make-Makefiles.bash

 ./make-Makefiles.bash 
-- cmake version 3.5.1
-- The C compiler identification is GNU 6.3.1
-- The CXX compiler identification is GNU 6.3.1
-- Check for working C compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc
-- Check for working C compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-g++
-- Check for working CXX compiler: /opt/gcc-aarch64-linux-gnu/bin/aarch64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_SYSTEM_PROCESSOR value `armv8-a` is unknown
-- Please add this value near 源码位置/mpp/CMakeLists.txt:102
-- Performing Test GCC_HAS_NO_NARROWING
-- Performing Test GCC_HAS_NO_NARROWING - Success
-- Performing Test GCC_HAS_STACK_REALIGN
-- Performing Test GCC_HAS_STACK_REALIGN - Failed
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28") 
-- Checking for one of the modules 'pthread'
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- compile with drm support
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    RKPLATFORM


-- Build files have been written to: 源码位置/mpp/build/linux/aarch64

双击使用cmake-gui打开/build/linux/aarch64/CMakeCache.txt
修改CMAKE_INSTALL_PREFIX为想要安装bin include lib的位置，本文使用/home/xzm/3-CSDN_Test_Project/1-mpp/install
点击Generate、Configure
执行make

./make
Scanning dependencies of target osal
[  0%] Building CXX object osal/CMakeFiles/osal.dir/mpp_platform.cpp.o
[  0%] Building CXX object osal/CMakeFiles/osal.dir/mpp_runtime.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_allocator.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_thread.cpp.o
[  1%] Building CXX object osal/CMakeFiles/osal.dir/mpp_common.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_queue.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_time.cpp.o
[  2%] Building CXX object osal/CMakeFiles/osal.dir/mpp_list.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_mem.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_env.cpp.o
[  3%] Building CXX object osal/CMakeFiles/osal.dir/mpp_log.cpp.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_allocator.c.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_mem.c.o
[  4%] Building C object osal/CMakeFiles/osal.dir/android/os_env.c.o
...
Scanning dependencies of target mpi_enc_multi_test
[ 99%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpi_enc_multi_test.c.o
[100%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpp_event_trigger.c.o
[100%] Building C object test/CMakeFiles/mpi_enc_multi_test.dir/mpp_parse_cfg.c.o
[100%] Linking CXX executable mpi_enc_multi_test
[100%] Built target mpi_enc_multi_test

执行make install

make install
[  9%] Built target osal
[ 10%] Built target mpp_device
[ 14%] Built target mpp_base
[ 14%] Built target vproc_iep
[ 15%] Built target mpp_rc
[ 16%] Built target codec_dummy_dec
[ 16%] Built target codec_avsd
...
[ 96%] Built target mpi_enc_test
[ 97%] Built target mpp_info_test
[ 98%] Built target mpi_test
[ 99%] Built target mpi_rc_test
[100%] Built target mpi_enc_multi_test
Install the project...
-- Install configuration: "Release"

查看安装文件夹/install

tree
.
├── bin
│   ├── mpi_dec_mt_test
│   ├── mpi_dec_multi_test
│   ├── mpi_dec_test
│   ├── mpi_enc_multi_test
│   ├── mpi_enc_test
│   ├── mpi_rc2_test
│   ├── mpi_rc_test
│   ├── mpi_test
│   ├── mpp_info_test
│   └── vpu_api_test
├── include
│   └── rockchip
│       ├── mpp_buffer.h
│       ├── mpp_err.h
│       ├── mpp_frame.h
│       ├── mpp_meta.h
│       ├── mpp_packet.h
│       ├── mpp_task.h
│       ├── rk_mpi_cmd.h
│       ├── rk_mpi.h
│       ├── rk_type.h
│       ├── rk_venc_cmd.h
│       ├── vpu_api.h
│       └── vpu.h
└── lib
    ├── librockchip_mpp.so -> librockchip_mpp.so.1
    ├── librockchip_mpp.so.0
    ├── librockchip_mpp.so.1 -> librockchip_mpp.so.0
    ├── librockchip_mpp_static.a
    ├── librockchip_vpu.so -> librockchip_vpu.so.1
    ├── librockchip_vpu.so.0
    ├── librockchip_vpu.so.1 -> librockchip_vpu.so.0
    ├── librockchip_vpu_static.a
    └── pkgconfig
        ├── rockchip_mpp.pc
        └── rockchip_vpu.pc

5 directories, 32 files

安装MPP结束

编译live555

下载live555源码:http://www.live555.com/liveMedia/public/
进入源码目录，新增config.rk3399（根据个人编译器指定内容），内容如下：

CROSS_COMPILE?=		aarch64-linux-gnu-
COMPILE_OPTS =		$(INCLUDES) -I. -O2 -DSOCKLEN_T=socklen_t -DNO_SSTREAM=1 -D_LARGEFILE_SOURCE=1 -D_FILE_OFFSET_BITS=64
C =			c
C_COMPILER =		$(CROSS_COMPILE)gcc
C_FLAGS =		$(COMPILE_OPTS)
CPP =			cpp
CPLUSPLUS_COMPILER =	$(CROSS_COMPILE)g++
CPLUSPLUS_FLAGS =	$(COMPILE_OPTS) -Wall -DBSD=1
OBJ =			o
LINK =			$(CROSS_COMPILE)g++ -o
LINK_OPTS =		
CONSOLE_LINK_OPTS =	$(LINK_OPTS)
LIBRARY_LINK =		$(CROSS_COMPILE)ar cr 
LIBRARY_LINK_OPTS =	$(LINK_OPTS)
LIB_SUFFIX =			a
LIBS_FOR_CONSOLE_APPLICATION =
LIBS_FOR_GUI_APPLICATION =
EXE =
#个人安装目录，自己修改
PREFIX = /home/xzm/3-CSDN_Test_Project/2-live555/install

运行genMakefiles生成makefile,根据后缀名匹配对应的config文件。

./genMakefiles rk3399

此时在当前文件夹下已经生成Makefile文件，我们需要进入相应的子文件夹修改一些prefix值,查找prefix的值，统统改为自己需要安装的地址即可。

#/home/xzm/3-CSDN_Test_Project/2-live555/live/BasicUsageEnvironment/Makefile
#其他几个文件夹也类似，修改之后进行make

make

make
cd liveMedia ; make
make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/liveMedia'
...
make[1]: Leaving directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'
For more information about this source code (including your obligations under the LGPL), please see our FAQ at http://live555.com/liveMedia/faq.html


make install

make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/liveMedia'
install -d /home/xzm/3-CSDN_Test_Project/2-live555/install/include/liveMedia /home/xzm/3-CSDN_Test_Project/2-live555/install/lib
install -m 644 include/*.hh /home/xzm/3-CSDN_Test_Project/2-live555/install/include/liveMedia
install -m 644 libliveMedia.a /home/xzm/3-CSDN_Test_Project/2-live555/install/lib
...
cd proxyServer ; make install
make[1]: Entering directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'
install -d /home/xzm/3-CSDN_Test_Project/2-live555/install/bin
install -m 755 live555ProxyServer /home/xzm/3-CSDN_Test_Project/2-live555/install/bin
make[1]: Leaving directory '/home/xzm/3-CSDN_Test_Project/2-live555/live/proxyServer'

live555编译完成

tree
.
├── bin
├── include
│   ├── BasicUsageEnvironment 
│   ├── groupsock
│   ├── liveMedia
│   └── UsageEnvironment
└── lib
    ├── libBasicUsageEnvironment.a
    ├── libgroupsock.a
    ├── libliveMedia.a
    └── libUsageEnvironment.a
7 directories, 213 files

部分demo源码记录

MPP 解码

具体详细的源码可以查看MPP官方文档的mpi_dec_test.c、mpi_dec_mt_test.c、mpi_dec_multi_test.c。

MPP_RET ret = mpi->decode_put_packet(ctx, packet); //put 很多次得到一个frame 输入h264的packet数据
MPP_RET ret = mpi->decode_get_frame(ctx, &frame); //得到解码的yuv数据

MPP 编码

具体详细的源码可以查看MPP官方文档的mpi_enc_test.c、mpi_enc_multi_test.c。

ret = mpi->encode_put_frame(ctx, frame); //阻塞函数，直到输入图像使用完成 输入yuv图像数据
ret = mpi->encode_get_packet(ctx, &packet_g);//得到编码的h264格式数据
// write packet to file here
 void *ptr   = mpp_packet_get_pos(packet_g); //得到数据指针位置
 size_t len  = mpp_packet_get_length(packet_g);//得到数据长度

 p->pkt_eos = mpp_packet_get_eos(packet_g);
 //copy数据并写至管道，供live555服务端使用
 void *cpy = malloc(len);
 memcpy(cpy, ptr, len);
 if(mpp_packet_write_to_fifo(cpy, len) < 0)//阻塞
 {
     // goto RET;
     printf(" mpp_packet_write_to_fifo err \n");
 }

live555利用管道实现h264实时视频RTSP推流

上面得到了h264编码后的文件后，写至管道，利用函数写入管道。

int mpp_packet_write_to_fifo(void *ptr, size_t len)
{  
    int ret = 0;
    if(!isOpen)
    {   
        printf("start open the fifo file\n");
        pipe_fd = open(fifo_name, O_WRONLY);    //阻塞至读端打开
        if(pipe_fd != -1)
        {
            printf("open fifo success\n");
            printf("thread mpp_packet_write_to_fifo, pipe_fd = %d\n", pipe_fd);
            isOpen = true;
            //return pipe_fd;
        }
        else
        {
            printf("pipe file open error %s\n", strerror(errno));
            return -1;
            // pthread_exit(NULL);
        }
    }
    ret = write(pipe_fd, ptr, len);
    if(ret != len)
    {
        printf("=======Write fifo Err======\n");
        return -1;
    }
    return 0;
}
//live555 接口 
    OutPacketBuffer::maxSize = 500000;//防止单帧数据过大
    
    char const* streamName = "stream";
    //char const* inputFileName = "my_fifo";
    char const* inputFileName = fifo_name;
    ServerMediaSession* sms = ServerMediaSession::createNew(*env, streamName, streamName, descriptionString);
    sms->addSubsession(H264VideoFileServerMediaSubsession::createNew(*env, inputFileName, reuseFirstSource));
    rtspServer->addServerMediaSession(sms);

live555源码分析

1.针对官方测试程序 live/testProgs/testOnDemandRTSPServer.cpp进行部分分析。

live/testProgs/testOnDemandRTSPServer.cpp main函数中创建服务端，并根据提供的文件名创建相应的类。

  // A H.264 video elementary stream:
  //创建h264RTSP服务端，播放的视频是out.h264
  {
    char const* streamName = "h264ESVideoTest";
    char const* inputFileName = "out.h264";//xzm0114
    ServerMediaSession* sms
      = ServerMediaSession::createNew(*env, streamName, streamName,
				      descriptionString);
    sms->addSubsession(H264VideoFileServerMediaSubsession
		       ::createNew(*env, inputFileName, reuseFirstSource));//根据文件名创建新的H264VideoFileServerMediaSubsession
    rtspServer->addServerMediaSession(sms);

    announceStream(rtspServer, sms, streamName, inputFileName);
  }

进一步查看live/liveMedia/H264VideoFileServerMediaSubsession.cpp

H264VideoFileServerMediaSubsession*
H264VideoFileServerMediaSubsession::createNew(UsageEnvironment& env,
					      char const* fileName,
					      Boolean reuseFirstSource) {
  return new H264VideoFileServerMediaSubsession(env, fileName, reuseFirstSource);
}

H264VideoFileServerMediaSubsession::H264VideoFileServerMediaSubsession(UsageEnvironment& env,
								       char const* fileName, Boolean reuseFirstSource)
  : FileServerMediaSubsession(env, fileName, reuseFirstSource),
    fAuxSDPLine(NULL), fDoneFlag(0), fDummyRTPSink(NULL) {
}

下一步live/liveMedia/FileServerMediaSubsession.cpp

FileServerMediaSubsession
::FileServerMediaSubsession(UsageEnvironment& env, char const* fileName,
			    Boolean reuseFirstSource)
  : OnDemandServerMediaSubsession(env, reuseFirstSource),
    fFileSize(0) {
    //头文件中定义 char const* fFileName; 子类可以通过继承得到文件名
  fFileName = strDup(fileName);
}

FileServerMediaSubsession::~FileServerMediaSubsession() {
  delete[] (char*)fFileName;
}

回到live/liveMedia/H264VideoFileServerMediaSubsession.cpp中

//createNewStreamSource 为虚函数 父类中进行使用
FramedSource* H264VideoFileServerMediaSubsession::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
  estBitrate = 500; // kbps, estimate

  // Create the video source:
  //xzm0114 通过继承得到的fFilename 创建ByteStreamFileSource
  ByteStreamFileSource* fileSource = ByteStreamFileSource::createNew(envir(), fFileName);
  if (fileSource == NULL) return NULL;
  fFileSize = fileSource->fileSize();

  // Create a framer for the Video Elementary Stream:
  return H264VideoStreamFramer::createNew(envir(), fileSource);
}

这里重点看一下继承关系：
class H264VideoFileServerMediaSubsession: public FileServerMediaSubsession
class FileServerMediaSubsession: public OnDemandServerMediaSubsession
打开 live/liveMedia/OnDemandServerMediaSubsession.cpp查看调用createNewStreamSource的地方，一共有两处。

OnDemandServerMediaSubsession::sdpLines() {
  if (fSDPLines == NULL) {
    // We need to construct a set of SDP lines that describe this
    // subsession (as a unicast stream).  To do so, we first create
    // dummy (unused) source and "RTPSink" objects,
    // whose parameters we use for the SDP lines:
    unsigned estBitrate;
    //调用子类实现的createNewStreamSource
    /* 头文件中定义虚函数
    protected: // new virtual functions, defined by all subclasses
  	virtual FramedSource* createNewStreamSource(unsigned clientSessionId,
					      unsigned& estBitrate) = 0;
	*/
    FramedSource* inputSource = createNewStreamSource(0, estBitrate);
    printf("=====xzm %s:%d:%s=======\n\n",__FILE__, __LINE__, __func__); //xzm0114
    if (inputSource == NULL) return NULL; // file not found

    struct in_addr dummyAddr;
    dummyAddr.s_addr = 0;
    Groupsock* dummyGroupsock = createGroupsock(dummyAddr, 0);
    unsigned char rtpPayloadType = 96 + trackNumber()-1; // if dynamic
    RTPSink* dummyRTPSink = createNewRTPSink(dummyGroupsock, rtpPayloadType, inputSource);
    if (dummyRTPSink != NULL && dummyRTPSink->estimatedBitrate() > 0) estBitrate = dummyRTPSink->estimatedBitrate();

    setSDPLinesFromRTPSink(dummyRTPSink, inputSource, estBitrate);
    Medium::close(dummyRTPSink);
    delete dummyGroupsock;
    closeStreamSource(inputSource);
    printf("=====xzm %s:%d:%s=======\n\n",__FILE__, __LINE__, __func__); //xzm0114

  }

  return fSDPLines;
}

//void OnDemandServerMediaSubsession::getStreamParameters函数中
 FramedSource* mediaSource
      = createNewStreamSource(clientSessionId, streamBitrate);
    // Create 'groupsock' and 'sink' objects for the destination,
    // using previously unused server port numbers:

重点讲一下文件打开和关闭的接口函数
live/liveMedia/InputFile.cpp中实现了OpenInputFile和CloseInputFile，并在live/liveMedia/ByteStreamFileSource.cpp中调用

FILE* OpenInputFile(UsageEnvironment& env, char const* fileName) {
  FILE* fid;

  // Check for a special case file name: "stdin"
  if (strcmp(fileName, "stdin") == 0) {
    fid = stdin;
#if (defined(__WIN32__) || defined(_WIN32)) && !defined(_WIN32_WCE)
    _setmode(_fileno(stdin), _O_BINARY); // convert to binary mode
#endif
  } else {
    fid = fopen(fileName, "rb");
    if (fid == NULL) {
      env.setResultMsg("unable to open file \"",fileName, "\"");
    }
    printf("\n\n\n\n\n======xzm Open file:%s========\n\n\n\n\n\n", fileName); //xzm0113

  }

  return fid;
}

void CloseInputFile(FILE* fid) {
  // Don't close 'stdin', in case we want to use it again later.
  if (fid != NULL && fid != stdin) fclose(fid);
  printf("\n\n\n\n\n======xzm Close file ========\n\n\n\n\n\n");//xzm0113
}

打开live/liveMedia/ByteStreamFileSource.cpp，函数根据不同的文件类型使用fread或者read，对于管道而言，函数会使用read进行读取。

void ByteStreamFileSource::doReadFromFile() {
  // Try to read as many bytes as will fit in the buffer provided (or "fPreferredFrameSize" if less)
  if (fLimitNumBytesToStream && fNumBytesToStream < (u_int64_t)fMaxSize) {
    fMaxSize = (unsigned)fNumBytesToStream;
  }
  if (fPreferredFrameSize > 0 && fPreferredFrameSize < fMaxSize) {
    fMaxSize = fPreferredFrameSize;
  }
#ifdef READ_FROM_FILES_SYNCHRONOUSLY
  fFrameSize = fread(fTo, 1, fMaxSize, fFid);
#else
  //xzm0113

  if (fFidIsSeekable) {
    printf("======Use fread=======\n\n");//xzm0113
    fFrameSize = fread(fTo, 1, fMaxSize, fFid);
  } else {
    // For non-seekable files (e.g., pipes), call "read()" rather than "fread()", to ensure that the read doesn't block:
    

    fFrameSize = read(fileno(fFid), fTo, fMaxSize);
    printf("======Use read,fFrameSize:%d ,fMaxSize:%d =======\n\n", fFrameSize, fMaxSize);//xzm0113
  }
#endif
  if (fFrameSize == 0) {
    handleClosure();
    return;
  }
  fNumBytesToStream -= fFrameSize;

  // Set the 'presentation time':
  if (fPlayTimePerFrame > 0 && fPreferredFrameSize > 0) {
    if (fPresentationTime.tv_sec == 0 && fPresentationTime.tv_usec == 0) {
      // This is the first frame, so use the current time:
      gettimeofday(&fPresentationTime, NULL);
    } else {
      // Increment by the play time of the previous data:
      unsigned uSeconds	= fPresentationTime.tv_usec + fLastPlayTime;
      fPresentationTime.tv_sec += uSeconds/1000000;
      fPresentationTime.tv_usec = uSeconds%1000000;
    }

    // Remember the play time of this data:
    fLastPlayTime = (fPlayTimePerFrame*fFrameSize)/fPreferredFrameSize;
    fDurationInMicroseconds = fLastPlayTime;
  } else {
    // We don't know a specific play time duration for this data,
    // so just record the current time as being the 'presentation time':
    gettimeofday(&fPresentationTime, NULL);
  }

  // Inform the reader that he has data:
#ifdef READ_FROM_FILES_SYNCHRONOUSLY
  // To avoid possible infinite recursion, we need to return to the event loop to do this:
  nextTask() = envir().taskScheduler().scheduleDelayedTask(0,
				(TaskFunc*)FramedSource::afterGetting, this);
#else
  // Because the file read was done from the event loop, we can call the
  // 'after getting' function directly, without risk of infinite recursion:
  FramedSource::afterGetting(this);
#endif
}

注：源码本人修改了一些定义的值，是为了避免单帧过大导致无法读取，具体有

// live/liveMedia/StreamParser.cpp
// #define BANK_SIZE 150000
#define BANK_SIZE 3000000 //xzm0113
OutPacketBuffer::maxSize = 500000;//1229xzm live555初始化时执行