ArmSoM Rockchip系列產品通用教程之Camera使用

1. Camera 簡介?

ArmSoM系列產品使用的是mipi-csi接口的攝像頭

ArmSoM-Sige7支持雙攝同顯：

2. RK3588硬件通路框圖?

rk3588支持2個isp硬件，每個isp設備可虛擬出多個虛擬節點，軟件上通過回讀的方式，依次從ddr讀取每一路的圖像數據進isp處理。對于多攝方案，建議將數據流平均分配到兩個isp上。

回讀：指數據經過vicap采集到ddr，應用獲取到數據后，將buffer地址推送給isp，isp再從ddr獲取圖像數據。

3. RK3588 的camera通路:?

多sensor支持：

• 單路硬件isp最多支持4路復用，isp復用情況支持分辨率如下：
• 2路復用：最大分辨率3840x2160，dts對應配置2路rkisp_vir設備。
• 3路或4路復用：最大分辨率2560x1536，dts對應配置3或4路rkisp_vir設備。
• 硬件支持最多采集7路sensor：6mipi + 1dvp，多sensor軟件通路如下：

下圖是RK3588 camera連接鏈路示意圖，可以支持7路camera。

4. 鏈路解析：?

• 圖中：mipi camera2---> csi2_dphy1 ---> mipi2_csi2 ---> rkcif_mipi_lvds2--->rkcif_mipi_lvds2_sditf --->rkisp0_vir2
• 對應節點：imx415 ---> csi2_dphy0 ---> mipi2_csi2 ---> rkcif_mipi_lvds2--->rkcif_mipi_lvds2_sditf --->rkisp0_vir2
• 鏈接關系：sensor---> csi2 dphy---->mipi csi host--->vicap
• 實線鏈路解析： Camera sensor ---> dphy ---> 通過mipi_csi2模塊解析mipi協議---> vicap ( rkcif節點代表vicap )
• 虛線鏈路解析：vicap ---> rkcif_mipi_lvds2_sditf ---> isp

提示

每個vicap節點與isp的鏈接關系，通過對應虛擬出的XXX_sditf來指明鏈接關系。

5. ArmSoM-Sige7雙路Camera調試?

這里以imx415攝像頭為例，解析ArmSoM-Sige7雙路Camera調試

5.1 原理圖?

Camera1：CSI0_MIPI：

Camera2： CS1_MIPI：

5.2 雙路Camera的dts配置：?

• 鏈路配置1： imx415 —> csi2_dphy0 —> mipi2_csi2 —> rkcif_mipi_lvds2—>rkcif_mipi_lvds2_sditf —>rkisp0_vir2
• 鏈路配置2： imx415 —> csi2_dphy3 —> mipi4_csi2 —> rkcif_mipi_lvds4—>rkcif_mipi_lvds4_sditf —>rkisp1_vir1

&i2c3 {    status = "okay";    imx415: imx415@1a {        status = "okay";        compatible = "sony,imx415";        reg = ;        clocks = ;        clock-names = "xvclk";        pinctrl-names = "default";        pinctrl-0 = ;        power-domains = ;        pwdn-gpios = ;        reset-gpios = ;        rockchip,camera-module-index = ;        rockchip,camera-module-facing = "back";        rockchip,camera-module-name = "CMK-OT2022-PX1";        rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20";        port {            imx415_out0: endpoint {                remote-endpoint = ;                data-lanes = ;            };        };    };};&i2c4 {    status = "okay";    pinctrl-names = "default";    pinctrl-0 = ;    imx415_1: imx415_1@1a {        status = "okay";        compatible = "sony,imx415";        reg = ;        clocks = ;        clock-names = "xvclk";        pinctrl-names = "default";        pinctrl-0 = ;        power-domains = ;        pwdn-gpios = ;        reset-gpios = ;        rockchip,camera-module-index = ;        rockchip,camera-module-facing = "back";        rockchip,camera-module-name = "CMK-OT2022-PX1";        rockchip,camera-module-lens-name = "IR0147-50IRC-8M-F20";        port {            imx415_out3: endpoint {                remote-endpoint = ;                data-lanes = ;            };        };    };};&csi2_dphy0_hw {    status = "okay";};&csi2_dphy1_hw {    status = "okay";};&csi2_dphy0 {    status = "okay";    ports {        #address-cells = ;        #size-cells = ;        port@0 {            reg = ;            #address-cells = ;            #size-cells = ;            mipidphy0_in_ucam0: endpoint@1 {                reg = ;                remote-endpoint = ;                data-lanes = ;            };        };        port@1 {            reg = ;            #address-cells = ;            #size-cells = ;            csidphy0_out: endpoint@0 {                reg = ;                remote-endpoint = ;            };        };    };};&csi2_dphy3 {    status = "okay";    ports {        #address-cells = ;        #size-cells = ;        port@0 {            reg = ;            #address-cells = ;            #size-cells = ;            mipidphy3_in_ucam3: endpoint@1 {                reg = ;                remote-endpoint = ;                data-lanes = ;            };        };        port@1 {            reg = ;            #address-cells = ;            #size-cells = ;            csidphy3_out: endpoint@0 {                reg = ;                remote-endpoint = ;            };        };    };};&mipi2_csi2 {    status = "okay";    ports {        #address-cells = ;        #size-cells = ;        port@0 {            reg = ;            #address-cells = ;            #size-cells = ;            mipi2_csi2_input: endpoint@1 {                reg = ;                remote-endpoint = ;            };        };        port@1 {            reg = ;            #address-cells = ;            #size-cells = ;            mipi2_csi2_output: endpoint@0 {                reg = ;                remote-endpoint = ;            };        };    };};&mipi4_csi2 {    status = "okay";    ports {        #address-cells = ;        #size-cells = ;        port@0 {            reg = ;            #address-cells = ;            #size-cells = ;            mipi4_csi2_input: endpoint@1 {                reg = ;                remote-endpoint = ;            };        };        port@1 {            reg = ;            #address-cells = ;            #size-cells = ;            mipi4_csi2_output: endpoint@0 {                reg = ;                remote-endpoint = ;            };        };    };};&rkcif {    status = "okay";};&rkcif_mipi_lvds2 {    status = "okay";    port {        cif_mipi2_in0: endpoint {            remote-endpoint = ;        };    };};&rkcif_mipi_lvds2_sditf {    status = "okay";    port {        mipi_lvds2_sditf: endpoint {            remote-endpoint = ;        };    };};&rkcif_mipi_lvds4 {    status = "okay";    port {        cif_mipi_in4: endpoint {            remote-endpoint = ;        };    };};&rkcif_mipi_lvds4_sditf {    status = "okay";    port {        mipi4_lvds_sditf: endpoint {            remote-endpoint = ;        };    };};&rkcif_mmu {    status = "okay";};&rkisp0 {    status = "okay";};&isp0_mmu {    status = "okay";};&rkisp0_vir2 {    status = "okay";    port {        #address-cells = ;        #size-cells = ;        isp0_vir2: endpoint@0 {            reg = ;            remote-endpoint = ;        };    };};&rkisp1 {    status = "okay";};&isp1_mmu {    status = "okay";};&rkisp1_vir1 {    status = "okay";    port {        #address-cells = ;        #size-cells = ;        isp1_vir1: endpoint@0 {            reg = ;            remote-endpoint = ;        };    };};&pinctrl {    camera {        cam_pwdn_gpio: cam-pwdn-gpio {            rockchip,pins = ;        };    };};

6. 調試技巧?

6.1 查看Camera是否掛載到i2c總線下?

i2cdetect -y 3

6.2 查看拓撲結構?

 media-ctl -d /dev/media0 -p

6.3 imx415 相關的log信息?

dmesg | grep imx415

6.4 查看sys文件系統中文件信息?

內核會為攝像頭在目錄/sys/class/video4linux下分配設備信息描述文件

armsom@armsom:~$  grep imx415 /sys/class/video4linux/v*/name/sys/class/video4linux/v4l-subdev2/name:m00_b_imx415 3-001a/sys/class/video4linux/v4l-subdev7/name:m01_b_imx415 4-001a

查找Camera對應的vedio節點：

armsom@armsom:~$ grep "" /sys/class/video4linux/v*/name | grep mainpath/sys/class/video4linux/video22/name:rkisp_mainpath/sys/class/video4linux/video31/name:rkisp_mainpath

可以看到，在ArmSoM-Sige7中，雙Camera的節點對應的是：video22和video31

6.5 查找所有攝像頭設備?

armsom@armsom:~$ v4l2-ctl --list-devicesrkisp-statistics (platform: rkisp):        /dev/video29        /dev/video30        /dev/video38        /dev/video39rkcif-mipi-lvds2 (platform:rkcif):        /dev/media0        /dev/media1rkcif (platform:rkcif-mipi-lvds2):        /dev/video0        /dev/video1        /dev/video2        /dev/video3        /dev/video4        /dev/video5        /dev/video6        /dev/video7        /dev/video8        /dev/video9        /dev/video10rkcif (platform:rkcif-mipi-lvds4):        /dev/video11        /dev/video12        /dev/video13        /dev/video14        /dev/video15        /dev/video16        /dev/video17        /dev/video18        /dev/video19        /dev/video20        /dev/video21rkisp_mainpath (platform:rkisp0-vir0):        /dev/video22        /dev/video23        /dev/video24        /dev/video25        /dev/video26        /dev/video27        /dev/video28        /dev/media2rkisp_mainpath (platform:rkisp1-vir1):        /dev/video31        /dev/video32        /dev/video33        /dev/video34        /dev/video35        /dev/video36        /dev/video37        /dev/media3

其中/dev/video22和/dev/video31都是攝像頭的設備。

6.6 查看設備的預覽支持格式?

如下是video22節點： imx415 攝像頭的查詢結果：

armsom@armsom:~$ v4l2-ctl -d /dev/video22 --list-formats-extioctl: VIDIOC_ENUM_FMT        Type: Video Capture Multiplanar        [0]: 'UYVY' (UYVY 4:2:2)                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [1]: 'NV16' (Y/CbCr 4:2:2)                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [2]: 'NV61' (Y/CrCb 4:2:2)                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [3]: 'NV21' (Y/CrCb 4:2:0)                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [4]: 'NV12' (Y/CbCr 4:2:0)                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [5]: 'NM21' (Y/CrCb 4:2:0 (N-C))                Size: Stepwise 32x32 - 3840x2160 with step 8/8        [6]: 'NM12' (Y/CbCr 4:2:0 (N-C))                Size: Stepwise 32x32 - 3840x2160 with step 8/8

6.7 查看設備的所有信息：?

armsom@armsom:~$ v4l2-ctl --all --device /dev/video22Driver Info:        Driver name      : rkisp_v6        Card type        : rkisp_mainpath        Bus info         : platform:rkisp0-vir0        Driver version   : 2.3.0        Capabilities     : 0x84201000                Video Capture Multiplanar                Streaming                Extended Pix Format                Device Capabilities        Device Caps      : 0x04201000                Video Capture Multiplanar                Streaming                Extended Pix FormatMedia Driver Info:        Driver name      : rkisp0-vir0        Model            : rkisp0        Serial           :        Bus info         :        Media version    : 5.10.160        Hardware revision: 0x00000000 (0)        Driver version   : 5.10.160Interface Info:        ID               : 0x03000007        Type             : V4L VideoEntity Info:        ID               : 0x00000006 (6)        Name             : rkisp_mainpath        Function         : V4L2 I/O        Pad 0x01000009   : 0: Sink          Link 0x0200000a: from remote pad 0x1000004 of entity 'rkisp-isp-subdev' (Unknown V4L2 Sub-Device): Data, EnabledPriority: 2Format Video Capture Multiplanar:        Width/Height      : 3840/2160        Pixel Format      : 'NM12' (Y/CbCr 4:2:0 (N-C))        Field             : None        Number of planes  : 2        Flags             :        Colorspace        : sRGB        Transfer Function : Rec. 709        YCbCr/HSV Encoding: Rec. 709        Quantization      : Full Range        Plane 0           :           Bytes per Line : 3840           Size Image     : 8294400        Plane 1           :           Bytes per Line : 3840           Size Image     : 4147200Selection Video Capture: crop, Left 0, Top 0, Width 3840, Height 2160, Flags:Selection Video Capture: crop_bounds, Left 0, Top 0, Width 3840, Height 2160, Flags:Selection Video Output: crop, Left 0, Top 0, Width 3840, Height 2160, Flags:Selection Video Output: crop_bounds, Left 0, Top 0, Width 3840, Height 2160, Flags:Image Processing Controls                     pixel_rate 0x009f0902 (int64)  : min=0 max=1000000000 step=1 default=1000000000 value=356800000 flags=read-only, volatile

6.8 攝像頭預覽?

ArmSoM-Sige7中，雙Camera的預覽命令：

• 預覽攝像頭1：

gst-launch-1.0 v4l2src device=/dev/video22 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! autovideosink

• 預覽攝像頭2：

gst-launch-1.0 v4l2src device=/dev/video31 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! autovideosink

7. Camera應用程序開發?

客戶可以根據自己的需求進行Camera相關的應用程序開發，如下是使用QT開發的雙攝同顯應用程序。