ICode9

精准搜索请尝试: 精确搜索
首页 > 其他分享> 文章详细

Android投屏(屏幕共享)设计需要考虑的关键因素

2020-06-27 16:41:13  阅读:297  来源: 互联网

标签:publisher cliped int 关键因素 height width 投屏 Android event


许多开发者,在做智慧教室同屏、会议同屏之类的方案时,基于Andriod平台的采集,往往遇到各种各样的问题,以下就几个点,抛砖引玉:

1. 内网环境下,组播还是RTMP?

回答:这个问题,被无数的开发者问到,为此,单独写了篇博客论证:https://blog.csdn.net/renhui1112/article/details/86741428,感兴趣的可以参考下,简单来说,能RTMP的,就RTMP,如果真是内网环境下,没有并发瓶颈的同屏,可以启动内置RTSP服务(走单播),然后,其他终端拉流也不失为一个好的方案。

2. 推送分辨率如何设定或缩放?

回答:一般来说,好多Android设备,特别是高分屏,拿到的视频原始宽高非常大,如果推原始分辨率,编码和上行压力大,所以,一般建议,适当缩放,比如宽高缩放至2/3,缩放一般建议等比例缩放,此外,缩放宽高建议16字节对齐。

废话不多说,上实例代码:

    private void createScreenEnvironment() {

        sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();
        screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();

        Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "
                + screenWindowHeight);

        if (sreenWindowWidth > 800)
        {
            if (screenResolution == SCREEN_RESOLUTION_STANDARD)
            {
                scale_rate = SCALE_RATE_HALF;
                sreenWindowWidth = align(sreenWindowWidth / 2, 16);
                screenWindowHeight = align(screenWindowHeight / 2, 16);
            }
            else if(screenResolution == SCREEN_RESOLUTION_LOW)
            {
                scale_rate = SCALE_RATE_TWO_FIFTHS;
                sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);

            }
        }

        Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);

        int pf = mWindowManager.getDefaultDisplay().getPixelFormat();
        Log.i(TAG, "display format:" + pf);

        DisplayMetrics displayMetrics = new DisplayMetrics();
        mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);
        mScreenDensity = displayMetrics.densityDpi;

        mImageReader = ImageReader.newInstance(sreenWindowWidth,
                screenWindowHeight, 0x1, 6);

        mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);
    }

3. 横竖屏自动适配

回答:因为横竖屏状态下,采集的屏幕宽高不一样,如果横竖屏切换,这个时候,需要考虑到横竖屏适配问题,确保比如竖屏状态下,切换到横屏时,推拉流两端可以自动适配,横竖屏自动适配,编码器需要重启,拉流端,需要能自动适配宽高变化,自动播放。

4. 一定的补帧策略

回答:好多人不理解为什么要补帧,实际上,屏幕采集的时候,屏幕不动的话,不会一直有数据下去,这个时候,比较好的做法是,保存最后一帧数据,设定一定的补帧间隔,确保不会因为帧间距太大,导致播放端几秒都收不到数据,当然,如果服务器可以缓存GOP,这个问题迎刃而解。

5. 异常网络处理、事件回调机制

回答:如果是走RTMP,网络抖动或者其他网络异常,需要有好重连机制和状态回馈机制。

    class EventHandeV2 implements NTSmartEventCallbackV2 {
        @Override
        public void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {

            Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);

            String publisher_event = "";

            switch (id) {
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:
                    publisher_event = "开始..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:
                    publisher_event = "连接中..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:
                    publisher_event = "连接失败..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:
                    publisher_event = "连接成功..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:
                    publisher_event = "连接断开..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:
                    publisher_event = "关闭..";
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:
                    publisher_event = "开始一个新的录像文件 : " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:
                    publisher_event = "已生成一个录像文件 : " + param3;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:
                    publisher_event = "发送时延: " + param1 + " 帧数:" + param2;
                    break;

                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:
                    publisher_event = "快照: " + param1 + " 路径:" + param3;

                    if (param1 == 0) {
                        publisher_event = publisher_event + "截取快照成功..";
                    } else {
                        publisher_event = publisher_event + "截取快照失败..";
                    }
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:
                    publisher_event = "RTSP服务URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:
                    publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;
                    break;
                case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:
                    publisher_event ="服务器不支持RTSP推送, 推送的RTSP URL: " + param3;
                    break;
            }

            String str = "当前回调状态:" + publisher_event;

            Log.i(TAG, str);

            Message message = new Message();
            message.what = PUBLISHER_EVENT_MSG;
            message.obj = publisher_event;
            handler.sendMessage(message);
        }
    }

6. 部分屏幕数据采集

回答:我们遇到的好多场景下,教室端,会拿出来3/4的区域用来投递给学生看,1/4的区域,用来做一些指令等操作,这个时候,就需要考虑屏幕区域裁剪,接口可做如下设计:

	/**
	 * 投递裁剪过的RGBA数据
	 *
	 * @param data: RGBA data
	 *
	 * @param rowStride: stride information
	 *
	 * @param width: width
	 *
	 * @param height: height
	 *
	 * @param clipedLeft: 左;  clipedTop: 上; clipedwidth: 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数
	 *
	 * @return {0} if successful
	 */
	public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle,  ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);
                        //实际裁剪比例,可酌情自行调整
                        int left = 100;
                        int cliped_left = 0;

                        int top = 0;
                        int cliped_top = 0;

                        int cliped_width = width_;
                        int cliped_height = height_;

                        if(scale_rate == SCALE_RATE_HALF)
                        {
                            cliped_left = left / 2;
                            cliped_top = top / 2;

                            //宽度裁剪后,展示3/4比例
                            cliped_width = (width_ *3)/4;
                            //高度不做裁剪
                            cliped_height = height_;
                        }
                        else if(scale_rate == SCALE_RATE_TWO_FIFTHS)
                        {
                            cliped_left = left * 2 / 5;
                            cliped_top = top * 2 / 5;

                            //宽度裁剪后,展示3/4比例
                            cliped_width = (width_ *3)/4;
                            //高度不做裁剪
                            cliped_height = height_;
                        }

                        if(cliped_width % 2 != 0)
                        {
                            cliped_width = cliped_width + 1;
                        }

                        if(cliped_height % 2 != 0)
                        {
                            cliped_height = cliped_height + 1;
                        }

                        if ( (cliped_left + cliped_width) > width_)
                        {
                            Log.e(TAG, " invalid cliped region settings, cliped_left: " + cliped_left + " cliped_width:" + cliped_width + " width:" + width_);

                            return;
                        }

                        if ( (cliped_top + cliped_height) > height_)
                        {
                            Log.e(TAG, "invalid cliped region settings, cliped_top: " + cliped_top + " cliped_height:" + cliped_height + " height:" + height_);

                            return;
                        }

                        //Log.i(TAG, " clipLeft: " + cliped_left + " clipTop: " + cliped_top +  " clipWidth: " + cliped_width + " clipHeight: " + cliped_height);

                        libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_,
                                width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );

7. 文字、图片水印

回答:好多场景下,同屏者会把公司logo,和一定的文字信息展示在推送端,这个时候,需要考虑到文字和图片水印问题,具体可参考如下接口设置:

   /**
     * Set Text water-mark(设置文字水印)
     * 
     * @param fontSize: it should be "MEDIUM", "SMALL", "BIG"
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting font water-mark when publishing stream. </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);
    
    
    /**
     * Set Text water-mark font file name(设置文字水印字体路径)
	 *
     * @param fontFileName:  font full file name, e.g: /system/fonts/DroidSansFallback.ttf
	 *
	 * @return {0} if successful
     */
    public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);
	
    /**
     * Set picture water-mark(设置png图片水印)
     * 											
     * @param picPath: the picture working path, e.g: /sdcard/logo.png
     * 
     * @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
     * 
     * @param picWidth, picHeight: picture width & height
     * 
     * @param xPading, yPading: the distance of the original picture.
     * 
     * <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre>  
     * 
     * @return {0} if successful
     */
    public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);

总结:其实一个好的同屏系统,需要考虑的地方远不止以上几点,比如编码参数策略等,都需要考量,后续有机会再和大家做进一步分享。

标签:publisher,cliped,int,关键因素,height,width,投屏,Android,event
来源: https://blog.csdn.net/renhui1112/article/details/106967568

本站声明: 1. iCode9 技术分享网(下文简称本站)提供的所有内容,仅供技术学习、探讨和分享;
2. 关于本站的所有留言、评论、转载及引用,纯属内容发起人的个人观点,与本站观点和立场无关;
3. 关于本站的所有言论和文字,纯属内容发起人的个人观点,与本站观点和立场无关;
4. 本站文章均是网友提供,不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属;如您发现该文章侵犯了您的权益,可联系我们第一时间进行删除;
5. 本站为非盈利性的个人网站,所有内容不会用来进行牟利,也不会利用任何形式的广告来间接获益,纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

专注分享技术,共同学习,共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有