网站建设询价,郑州官网网站推广优化,影视免费网站模板,知名的网站建设公司排名背景
本文主要讲的是基于Android平台实现RTMP的技术方案设计#xff0c;基础架构图如下#xff1a; 组网注意事项
1. 组网#xff1a;无线组网#xff0c;需要好的AP模块才能撑得住大的并发流量#xff0c;推送端到AP#xff0c;最好是有线网链接#xff1b;
2. 服务…背景
本文主要讲的是基于Android平台实现RTMP的技术方案设计基础架构图如下 组网注意事项
1. 组网无线组网需要好的AP模块才能撑得住大的并发流量推送端到AP最好是有线网链接
2. 服务器部署SRS或NGINX服务器可以和Windows平台的教师机部署在一台机器
3. 教师端如教师有移动的PAD可以直接推到RTMP服务器然后共享出去
4. 学生端直接拉取服务端的RTMP流播放即可
5. 教师和学生互动学生端如需作为示范案例屏幕数据共享给其他同学只需请求同屏数据反推到RTMP服务器其他学生查看即可。
6. 扩展监控如果需要更进一步的技术方案如教师端想监控学生端的屏幕情况可以有两种方案如学生端直接推RTMP过来或者学生端启动内置RTSP服务教师端想看的时候随时看即可亦可轮询播放。
Android端对接
推送分辨率如何设定或缩放
Android设备特别是高分屏拿到的视频原始宽高非常大如果推原始分辨率编码和上行压力大所以一般建议适当缩放比如宽高缩放至2/3缩放一般建议等比例缩放缩放宽高建议16字节对齐。 private void createScreenEnvironment() {sreenWindowWidth mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, screenWindowWidth: sreenWindowWidth ,screenWindowHeight: screenWindowHeight);if (sreenWindowWidth 800){if (screenResolution SCREEN_RESOLUTION_STANDARD){scale_rate SCALE_RATE_HALF;sreenWindowWidth align(sreenWindowWidth / 2, 16);screenWindowHeight align(screenWindowHeight / 2, 16);}else if(screenResolution SCREEN_RESOLUTION_LOW){scale_rate SCALE_RATE_TWO_FIFTHS;sreenWindowWidth align(sreenWindowWidth * 2 / 5, 16);}}Log.i(TAG, After adjust mWindowWidth: sreenWindowWidth , mWindowHeight: screenWindowHeight);int pf mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, display format: pf);DisplayMetrics displayMetrics new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity displayMetrics.densityDpi;mImageReader ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}
横竖屏自动适配
横竖屏状态下采集的屏幕宽高不一样如果横竖屏切换这个时候需要考虑到横竖屏适配问题确保比如竖屏状态下切换到横屏时推拉流两端可以自动适配横竖屏自动适配编码器需要重启拉流端需要能自动适配宽高变化自动播放。 public void onConfigurationChanged(Configuration newConfig) {try {super.onConfigurationChanged(newConfig);if (this.getResources().getConfiguration().orientation Configuration.ORIENTATION_LANDSCAPE) {Log.i(TAG, onConfigurationChanged cur: LANDSCAPE);} else if (this.getResources().getConfiguration().orientation Configuration.ORIENTATION_PORTRAIT) {Log.i(TAG, onConfigurationChanged cur: PORTRAIT);}if(isPushingRtmp || isRecording || isRTSPPublisherRunning){stopScreenCapture();clearAllImages();createScreenEnvironment();setupVirtualDisplay();}} catch (Exception ex) {}}
补帧策略
好多人不理解为什么要补帧实际上屏幕采集的时候屏幕不动的话不会一直有数据下去这个时候比较好的做法是保存最后一帧数据设定一定的补帧间隔确保不会因为帧间距太大导致播放端几秒都收不到数据当然如果服务器可以缓存GOP这个问题迎刃而解。
异常网络处理、事件回调机制
回答如果是走RTMP网络抖动或者其他网络异常需要有好重连机制和状态回馈机制。 class EventHandeV2 implements NTSmartEventCallbackV2 {Overridepublic void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {Log.i(TAG, EventHandeV2: handle handle id: id);String publisher_event ;switch (id) {case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:publisher_event 开始..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:publisher_event 连接中..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:publisher_event 连接失败..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:publisher_event 连接成功..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:publisher_event 连接断开..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:publisher_event 关闭..;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:publisher_event 开始一个新的录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:publisher_event 已生成一个录像文件 : param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:publisher_event 发送时延: param1 帧数: param2;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:publisher_event 快照: param1 路径 param3;if (param1 0) {publisher_event publisher_event 截取快照成功..;} else {publisher_event publisher_event 截取快照失败..;}break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:publisher_event RTSP服务URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:publisher_event RTSP status code received, codeID: param1 , RTSP URL: param3;break;case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:publisher_event 服务器不支持RTSP推送, 推送的RTSP URL: param3;break;}String str 当前回调状态 publisher_event;Log.i(TAG, str);Message message new Message();message.what PUBLISHER_EVENT_MSG;message.obj publisher_event;handler.sendMessage(message);}}
部分屏幕数据采集
回答我们遇到的好多场景下教室端会拿出来3/4的区域用来投递给学生看1/4的区域用来做一些指令等操作这个时候就需要考虑屏幕区域裁剪 /*** 投递裁剪过的RGBA数据** param data: RGBA data** param rowStride: stride information** param width: width** param height: height** param clipedLeft: 左; clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数** return {0} if successful*/public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);文字、图片水印
好多场景下同屏者会把公司logo和一定的文字信息展示在推送端这个时候需要考虑到文字和图片水印问题 /*** Set Text water-mark(设置文字水印)* * param fontSize: it should be MEDIUM, SMALL, BIG* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting font water-mark when publishing stream. /pre * * return {0} if successful*/public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);/*** Set Text water-mark font file name(设置文字水印字体路径)** param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf** return {0} if successful*/public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);/*** Set picture water-mark(设置png图片水印)* * param picPath: the picture working path, e.g: /sdcard/logo.png* * param waterPostion: it should be TOPLEFT, TOPRIGHT, BOTTOMLEFT, BOTTOMRIGHT.* * param picWidth, picHeight: picture width height* * param xPading, yPading: the distance of the original picture.* * pre The interface is only used for setting picture(logo) water-mark when publishing stream, with *.png format /pre * * return {0} if successful*/public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);
屏幕权限获取|数据采集
采集推送之前需要获取屏幕权限拿到屏幕数据后调用SDK接口完成推送或录像操作即可 TargetApi(Build.VERSION_CODES.LOLLIPOP)private boolean startScreenCapture() {Log.i(TAG, startScreenCapture..);setupMediaProjection();setupVirtualDisplay();return true;}private int align(int d, int a) {return (((d) (a - 1)) ~(a - 1));}SuppressWarnings(deprecation)SuppressLint(NewApi)private void createScreenEnvironment() {sreenWindowWidth mWindowManager.getDefaultDisplay().getWidth();screenWindowHeight mWindowManager.getDefaultDisplay().getHeight();Log.i(TAG, screenWindowWidth: sreenWindowWidth ,screenWindowHeight: screenWindowHeight);if (sreenWindowWidth 800){if (screenResolution SCREEN_RESOLUTION_STANDARD){scale_rate SCALE_RATE_HALF;sreenWindowWidth align(sreenWindowWidth / 2, 16);screenWindowHeight align(screenWindowHeight / 2, 16);}else if(screenResolution SCREEN_RESOLUTION_LOW){scale_rate SCALE_RATE_TWO_FIFTHS;sreenWindowWidth align(sreenWindowWidth * 2 / 5, 16);screenWindowHeight align(screenWindowHeight * 2 / 5, 16);}}Log.i(TAG, After adjust mWindowWidth: sreenWindowWidth , mWindowHeight: screenWindowHeight);int pf mWindowManager.getDefaultDisplay().getPixelFormat();Log.i(TAG, display format: pf);DisplayMetrics displayMetrics new DisplayMetrics();mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);mScreenDensity displayMetrics.densityDpi;mImageReader ImageReader.newInstance(sreenWindowWidth,screenWindowHeight, 0x1, 6);mMediaProjectionManager (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);}SuppressLint(NewApi)private void setupMediaProjection() {mMediaProjection mMediaProjectionManager.getMediaProjection(MainActivity.mResultCode, MainActivity.mResultData);}SuppressLint(NewApi)private void setupVirtualDisplay() {mVirtualDisplay mMediaProjection.createVirtualDisplay(ScreenCapture, sreenWindowWidth, screenWindowHeight,mScreenDensity,DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,mImageReader.getSurface(), null, null);mImageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {Overridepublic void onImageAvailable(ImageReader reader) {Image image mImageReader.acquireLatestImage();if (image ! null) {processScreenImage(image);//image.close();}}}, null);}private void startRecorderScreen() {Log.i(TAG, start recorder screen..);if (startScreenCapture()) {new Thread() {Overridepublic void run() {Log.i(TAG, start record..);}}.start();}}private ByteBuffer deepCopy(ByteBuffer source) {int sourceP source.position();int sourceL source.limit();ByteBuffer target ByteBuffer.allocateDirect(source.remaining());target.put(source);target.flip();source.position(sourceP);source.limit(sourceL);return target;}/*** Process image data as desired.*/SuppressLint(NewApi)private void processScreenImage(Image image) {if(!isPushingRtmp !isRecording !isRTSPPublisherRunning){image.close();return;}/*final Image.Plane[] planes image.getPlanes();width_ image.getWidth();height_ image.getHeight();row_stride_ planes[0].getRowStride();ByteBuffer buf deepCopy(planes[0].getBuffer());*/// Log.i(OnScreenImage, new image);pushImage(image);}SuppressLint(NewApi)private void stopScreenCapture() {if (mVirtualDisplay ! null) {mVirtualDisplay.release();mVirtualDisplay null;}}基础初始化 private void InitAndSetConfig() {//开始要不要采集音频或视频请自行设置publisherHandle libPublisher.SmartPublisherOpen(this.getApplicationContext(),audio_opt, video_opt, sreenWindowWidth,screenWindowHeight);if ( publisherHandle 0 ){return;}Log.i(TAG, publisherHandle publisherHandle);libPublisher.SetSmartPublisherEventCallbackV2(publisherHandle, new EventHandeV2());if(videoEncodeType 1){int h264HWKbps setHardwareEncoderKbps(true, sreenWindowWidth,screenWindowHeight);Log.i(TAG, h264HWKbps: h264HWKbps);int isSupportH264HWEncoder libPublisher.SetSmartPublisherVideoHWEncoder(publisherHandle, h264HWKbps);if (isSupportH264HWEncoder 0) {Log.i(TAG, Great, it supports h.264 hardware encoder!);}}else if (videoEncodeType 2){int hevcHWKbps setHardwareEncoderKbps(false, sreenWindowWidth,screenWindowHeight);Log.i(TAG, hevcHWKbps: hevcHWKbps);int isSupportHevcHWEncoder libPublisher.SetSmartPublisherVideoHevcHWEncoder(publisherHandle, hevcHWKbps);if (isSupportHevcHWEncoder 0) {Log.i(TAG, Great, it supports hevc hardware encoder!);}}if(is_sw_vbr_mode){int is_enable_vbr 1;int video_quality CalVideoQuality(sreenWindowWidth,screenWindowHeight, true);int vbr_max_bitrate CalVbrMaxKBitRate(sreenWindowWidth,screenWindowHeight);libPublisher.SmartPublisherSetSwVBRMode(publisherHandle, is_enable_vbr, video_quality, vbr_max_bitrate);}//音频相关可以参考SmartPublisher工程/*if (!is_speex){// set AAC encoderlibPublisher.SmartPublisherSetAudioCodecType(publisherHandle, 1);}else{// set Speex encoderlibPublisher.SmartPublisherSetAudioCodecType(publisherHandle, 2);libPublisher.SmartPublisherSetSpeexEncoderQuality(publisherHandle, 8);}libPublisher.SmartPublisherSetNoiseSuppression(publisherHandle, is_noise_suppression ? 1: 0);libPublisher.SmartPublisherSetAGC(publisherHandle, is_agc ? 1 : 0);*/// libPublisher.SmartPublisherSetClippingMode(publisherHandle, 0);//libPublisher.SmartPublisherSetSWVideoEncoderProfile(publisherHandle, sw_video_encoder_profile);//libPublisher.SmartPublisherSetSWVideoEncoderSpeed(publisherHandle, sw_video_encoder_speed);// libPublisher.SetRtmpPublishingType(publisherHandle, 0);libPublisher.SmartPublisherSetFPS(publisherHandle, 18); //帧率可调libPublisher.SmartPublisherSetGopInterval(publisherHandle, 18*3);//libPublisher.SmartPublisherSetSWVideoBitRate(publisherHandle, 1200, 2400); //针对软编码有效一般最大码率是平均码率的二倍libPublisher.SmartPublisherSetSWVideoEncoderSpeed(publisherHandle, 3);//libPublisher.SmartPublisherSaveImageFlag(publisherHandle, 1);}
准备推送|录像|启动RTSP服务 SuppressWarnings(deprecation)Overridepublic void onStart(Intent intent, int startId) {// TODO Auto-generated method stubsuper.onStart(intent, startId);Log.i(TAG, onStart);if (libPublisher null)return;clearAllImages();screenResolution intent.getExtras().getInt(SCREENRESOLUTION);videoEncodeType intent.getExtras().getInt(VIDEOENCODETYPE);push_type intent.getExtras().getInt(PUSHTYPE);Log.i(TAG, push_type: push_type);mWindowManager (WindowManager) getSystemService(Service.WINDOW_SERVICE);// 窗口管理者createScreenEnvironment();startRecorderScreen();//如果同时推送和录像设置一次就可以InitAndSetConfig();if ( publisherHandle 0 ){stopScreenCapture();return;}if(push_type PUSH_TYPE_RTMP){String publishURL intent.getStringExtra(PUBLISHURL);Log.i(TAG, publishURL: publishURL);if (libPublisher.SmartPublisherSetURL(publisherHandle, publishURL) ! 0) {stopScreenCapture();Log.e(TAG, Failed to set publish stream URL..);if (publisherHandle ! 0) {if (libPublisher ! null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle 0;}}return;}}//启动传递数据线程post_data_thread new Thread(new DataRunnable());Log.i(TAG, new post_data_thread..);is_post_data_thread_alive true;post_data_thread.start();//录像相关is_need_local_recorder intent.getExtras().getBoolean(RECORDER);if(is_need_local_recorder){ConfigRecorderParam();int startRet libPublisher.SmartPublisherStartRecorder(publisherHandle);if( startRet ! 0 ){isRecording false;Log.e(TAG, Failed to start recorder..);}else{isRecording true;}}//录像相关——if(push_type PUSH_TYPE_RTMP){Log.i(TAG, RTMP Pusher mode..);//推流相关int startRet libPublisher.SmartPublisherStartPublisher(publisherHandle);if (startRet ! 0) {isPushingRtmp false;Log.e(TAG, Failed to start push rtmp stream..);return;}else{isPushingRtmp true;}//推流相关--}else if(push_type PUSH_TYPE_RTSP){Log.i(TAG, RTSP Internal Server mode..);rtsp_handle_ libPublisher.OpenRtspServer(0);if (rtsp_handle_ 0) {Log.e(TAG, 创建rtsp server实例失败! 请检查SDK有效性);} else {int port 8554;if (libPublisher.SetRtspServerPort(rtsp_handle_, port) ! 0) {libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ 0;Log.e(TAG, 创建rtsp server端口失败! 请检查端口是否重复或者端口不在范围内!);}//String user_name admin;//String password 12345;//libPublisher.SetRtspServerUserNamePassword(rtsp_handle_, user_name, password);if (libPublisher.StartRtspServer(rtsp_handle_, 0) 0) {Log.i(TAG, 启动rtsp server 成功!);} else {libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ 0;Log.e(TAG, 启动rtsp server失败! 请检查设置的端口是否被占用!);return;}isRTSPServiceRunning true;}if(isRTSPServiceRunning){Log.i(TAG, onClick start rtsp publisher..);String rtsp_stream_name stream1;libPublisher.SetRtspStreamName(publisherHandle, rtsp_stream_name);libPublisher.ClearRtspStreamServer(publisherHandle);libPublisher.AddRtspStreamServer(publisherHandle, rtsp_handle_, 0);if (libPublisher.StartRtspStream(publisherHandle, 0) ! 0) {Log.e(TAG, 调用发布rtsp流接口失败!);return;}isRTSPPublisherRunning true;}}//如果同时推送和录像Audio启动一次就可以了CheckInitAudioRecorder();Log.i(TAG, onStart--);}private void stopPush() {if(!isPushingRtmp){return;}if (!isRecording !isRTSPPublisherRunning) {if (audioRecord_ ! null) {Log.i(TAG, stopPush, call audioRecord_.StopRecording..);audioRecord_.Stop();if (audioRecordCallback_ ! null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ null;}audioRecord_ null;}}if (libPublisher ! null) {libPublisher.SmartPublisherStopPublisher(publisherHandle);}if (!isRecording !isRTSPPublisherRunning) {if (publisherHandle ! 0) {if (libPublisher ! null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle 0;}}}}
停止推送|录像|RTSP服务 private void stopRecorder() {if(!isRecording){return;}if (!isPushingRtmp !isRTSPPublisherRunning) {if (audioRecord_ ! null) {Log.i(TAG, stopRecorder, call audioRecord_.StopRecording..);audioRecord_.Stop();if (audioRecordCallback_ ! null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ null;}audioRecord_ null;}}if (libPublisher ! null) {libPublisher.SmartPublisherStopRecorder(publisherHandle);}if (!isPushingRtmp !isRTSPPublisherRunning) {if (publisherHandle ! 0) {if (libPublisher ! null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle 0;}}}}//停止发布RTSP流private void stopRtspPublisher() {if(!isRTSPPublisherRunning){return;}if (!isPushingRtmp !isRecording) {if (audioRecord_ ! null) {Log.i(TAG, stopRtspPublisher, call audioRecord_.StopRecording..);audioRecord_.Stop();if (audioRecordCallback_ ! null) {audioRecord_.RemoveCallback(audioRecordCallback_);audioRecordCallback_ null;}audioRecord_ null;}}if (libPublisher ! null) {libPublisher.StopRtspStream(publisherHandle);}if (!isPushingRtmp !isRecording) {if (publisherHandle ! 0) {if (libPublisher ! null) {libPublisher.SmartPublisherClose(publisherHandle);publisherHandle 0;}}}}//停止RTSP服务private void stopRtspService() {if(!isRTSPServiceRunning){return;}if (libPublisher ! null rtsp_handle_ ! 0) {libPublisher.StopRtspServer(rtsp_handle_);libPublisher.CloseRtspServer(rtsp_handle_);rtsp_handle_ 0;}}感兴趣的开发者可酌情参考。
