MOT (Beyond Pixels: Leveraging Geometry and Shape Cues for Online Multi-Object Tracking)demo阅读_ 沐雨的博客-程序员宝宝

变量:

  • all_wts:3D-3D,3D-2D和2D-2D权重
    global_id:车辆总数
    K_all:相机内参
    angleMapdetectionsQ_dnos:原帧车辆回归框号
    detectionsT_dnos:下一帧车辆回归框号
    result_for_kitti_eval:预估结果
    total_detecions总检测数
    params_carCuboid(avgCar_Sz, sz_ub,sz_lb):车辆参数
    sz_lb:车参上界
    sz_ub:车参下界
    avgCar_Sz[l, h, w]:车参:长,宽,高
    params_2D3D      = struct( K,  n, h);
    K:相机内参
    n[0,1,0]:道路平面
    h:摄像机离地高度
    detection       = struct( 'dno',      -1.0,    ...  
                      'bbox',       zeros(4,1),   ...
                      'sigma_2D',    zeros(3,3),   ...
                      'yaw',        0.0,           ... 
                      'bvolume',    [],            ... 
                      'bvolume_proj',   [],        ...   
                      'k',              [],        ... 
                      'origin',      zeros(3,1),   ... 
                      'start_frame',    -1.0,      ... 
                      'end_frame',      -1.0,      ... 
                      'sigma_3D',       zeros(4,4)    ...                               
     );
    dno:车辆号
    bbox:车辆检测区域
    sigma_2D:2D激活函数
    yaw:偏航角
    bvolume:3D边界体积(bvolume),它是将偏航和LBH错误应用于平均汽车长方体边界后得到的点的凸包bvolume_projk:边界体积凸包
    origin:车辆位置原点
    start_frame:起始帧
    end_frame:终止帧
    sigma_3D:3D激活函数
    num_matches:匹配数
    pose:Monocular ORBSLAM获取参数[平移量,旋转量,偏航量]
    detections:车辆检测矩阵
    feautes_2D2D:图像特征矩阵
    start_fno:起始帧号
    end_fno:终止帧号
    scoreMatrices_for_seq(end_fno, 1)当前帧得分矩阵
    first_time:是否是首帧
    num_detectionsQ:当前帧检测出的车辆数
    num_detectionsT:下一帧检测出的车辆数
    detectionsQ:当前帧检测出的车辆位置回归框
    detectionsT:下一帧检测出的车辆位置回归框
    motion[1*4]:帧移动量

     

函数

  • getCanonicalCuboid(avg_sz):在图像二维图像上生成长方体
    getOffsetBasedOnYaw([车宽,车高],yaw):计算要应用到X的偏移量,即回归框底线的中心,以实际到达车辆的中心。此偏移只能应用于Z方向。 
    generateScoreMatrices(detectionsQ, detectionsT):生成得分矩阵
    get3D3DMatchScore(detectionsT{j}.bvolume, detectionsQ{i}.bvolume):3D车辆物体相交程度
    get3D2DMatchScore(detectionsT{j}.bbox, detectionsQ{i}.bvolume_proj):3d投影相交程度
    generate2D2DScoreMatrix(features_2D2D{i}, features_2D2D{i+1}):车辆回归框特征相似度矩阵

     

初始化权重,车参,结构体,导入pose矩阵,bounding box矩阵,feature矩阵,angle_map矩阵:

addpath('../third_party');
         % 33  32  22  vp
all_wts = [0.6 0.4 0.2 0.0];
       
dcolors = distinguishable_colors(800);

for ww = 1:size(all_wts,1)
    
    c33 = all_wts(ww,1);
    c32 = all_wts(ww,2);
    c22 = all_wts(ww,3);
    cvp = all_wts(ww,4);       
            
    % 道路平面和相机高度
    n =  [0; 1; 0];
    h =  1.72;
    
    % 平均车参    avgCar_Sz = [4.3; 2; 2];
    sz_ub     = [34; 31.5; 31.5];
    sz_lb     = [-34; -31.5; -31.5];
    
    K_all     = load('../Data/calib/calib_all_test.txt');
    
    
    angleMap = containers.Map([ 90:270 90:-1:0 360:-1:270], [0:-1:-180 0:90 90:180]);        
    
    hungarian_association = false;      
    detectionsQ_dnos = [];
    detectionsT_dnos = [];
    
    for s = [2]        
        
        seqNo = s;                                        
    
        result_for_kitti_eval = [];
        
        total_detecions = 0;
        
        %% SETUP PARAMETERS

        image_path    = sprintf('../Data/images/image_02/test/%04d',seqNo);      
        pose_path     = sprintf('../Data/ORBSLAM_pose/test/%04d/KITTITrajectoryComplete_new',seqNo);              
        K             = [ K_all(seqNo+1,1:3); K_all(seqNo+1,5:7); K_all(seqNo+1,9:11)];
                
        % 定义车辆参数,相机参数和车辆结构体
        params_carCuboid = struct('avg_Sz',         avgCar_Sz,    ...
                                  'sz_ub',          sz_ub,        ...
                                  'sz_lb',          sz_lb         ...
                                  );
        
        params_2D3D      = struct('K',              K,            ...
                                  'n',              n,            ...
                                  'h',              h             ...
                                  );
        
        detection       = struct( 'dno',           -1.0,         ...       
                                 'bbox',           zeros(4,1),   ...
                                 'sigma_2D',       zeros(3,3),   ...       
                                 'yaw',            0.0,          ...
                                 'bvolume',        [],           ...       
                                 'bvolume_proj',   [],           ...       
                                 'k',              [],           ...       
                                 'origin',         zeros(3,1),   ...
                                 'start_frame',    -1.0,         ...
                                 'end_frame',      -1.0,         ...
                                 'sigma_3D',       zeros(4,4)    ...
                                );
        
        
        
        num_matches = 0;
        num_false_positive = 0;
                
        
        %% SETUP STRUCTS
        global_id = 1;
        
        %% 导入数据 :
        
        % load pose (Monocular ORBSLAM)
        pose = load(pose_path);      
                                
        %load all detections cell array - variable name 'detections'
        load(sprintf('../Data/RRC_Detections_mat/test/%04d/detections_rrc_test_%02d.mat', seqNo,seqNo));                
        
        % feautes_2D2D
        load(sprintf('../Data/Features2D_mat/test/%04d/features_2_rrc_test_%02d.mat', seqNo,seqNo));               
        
        start_fno 		 = 1;
        end_fno 		 = 35;%size(pose,1); 
        
        scoreMatrices_for_seq = cell(end_fno, 1);
        
        %% RUN SCORING
        
        first_time = 1;
        tempDQ = [];
        
        figure(1);
        
        for i = 1:end_fno - 1
            
            disp(sprintf('Seq<%02d> | frame : %04d', seqNo, i));
            
            if(first_time)
                num_detectionsQ = size(detections{i},1);
            end
            
            num_detectionsT = size(detections{i+1},1);

如果输入是图像第一帧,将每个车辆位置,车辆序号,车辆偏航角,车辆3D转2D举证填入结构体,因为是第一帧,每填入一辆车,车号加一:

if(first_time)
        
                % load the detections in to the query and train struct cell arrays x1 y1 x2 y2 confidence ID
                for j = 1:num_detectionsQ
                    
                    detectionsQ{j} = detection; % emtpy detection struct
                    
                    detectionsQ{j}.dno          = global_id;
                    detectionsQ{j}.bbox         = detections{i}(j,1:4);
                    detectionsQ{j}.yaw          = deg2rad(-90) ;                   
                    detectionsQ{j}.sigma_3D     = [1.3 0 0 0; 0 1.1 0 0; 0 0 1.1 0; 0 0 0 deg2rad(0)];

                    
                    global_id = global_id + 1;  % increment the global ID
                end
            end

非第一帧,将当前帧下一帧的检测出来的所有车辆序号置一,填入当前帧下一帧的位置填入,偏航角置为90°,填入3D激活函数:

for j = 1:num_detectionsT
                
                detectionsT{j} = detection; % emtpy detection struct
                
                detectionsT{j}.dno          = -1;     % will be given after association
                detectionsT{j}.bbox         = detections{i+1}(j,1:4);
                detectionsT{j}.yaw  =       deg2rad(-90) ;
                detectionsT{j}.sigma_3D     = [1.3 0 0 0; 0 1.1 0 0; 0 0 1.1 0; 0 0 0 deg2rad(0)];;%
                %1.6, 1.1, 1.1 : 0.8 0.1 0.1
            end

计算两帧之间摄像头的平移,偏航角,旋转角变化,将θ置为0,然后将motion归一化:

 motion = [-(pose(i+1,11)-pose(i,11));
                      -(pose(i+1,12)-pose(i,12));
                      -(pose(i+1,13)-pose(i,13));
                        deg2rad(0)];
                        
            % scale the motion
            motion(1:3,1) =  motion(1:3,1).*(1.72/44);

将前后两帧的结构体,车参,相机参数,摄像头移动参数输入,完善结构体数据:

 [detectionsQ, detectionsT] = propagateDetections(detectionsQ,       ...
                                                             detectionsT,       ...
                                                             params_2D3D,       ...
                                                             params_carCuboid,  ...
                                                             motion             ...
                                                             );

其中,propagateDetections函数将2D图像转成3D,公式为X_{f}=\pi _{G}^{-1}(x)=hK^{-1}x/n{T}K^{-1}x

for i = 1:length(detectionsQ)                        
        
        b1Q = [];
        B1Q = [];
        bvolumeQ1 = [];
        
        % check if the detection is first time i.e. we have to start with
        % the canonical cuboid. Else we already have the cuboid.
        
        if(length(detectionsQ{i}.bvolume) == 0 )
            %获取标准长方体
            canonicalCuboid = getCanonicalCuboid( params_carCuboid.avg_Sz);   

            % 找寻底部中心
            b1Q = [detectionsQ{i}.bbox(1) + (detectionsQ{i}.bbox(3) - detectionsQ{i}.bbox(1))/2 ;
                  detectionsQ{i}.bbox(4);
                  1.0];

            % 投影成3d
            B1Q = (params_2D3D.h * inv(params_2D3D.K) * b1Q) / (params_2D3D.n' * inv(params_2D3D.K) * b1Q);

            % 应用偏移量,这是偏航的一个函数,并获得汽车的原点(记住近似值)
            offset_Z = getOffsetBasedOnYaw([params_carCuboid.avg_Sz(1); params_carCuboid.avg_Sz(3)], detectionsQ{i}.yaw);        

            % 当前帧中的汽车原点
            B1Q = B1Q + [0; 0; offset_Z];

            % 转换成标准长方体
            canonicalCuboid = canonicalCuboid + repmat(B1Q', 8,1);

            % 获取当前帧边界体积                
            [bvolumeQ1, k1] = getBoundingVolume(B1Q, canonicalCuboid, detectionsQ{i}.yaw, ... 
                                                detectionsQ{i}.sigma_3D, ...
                                                params_carCuboid.sz_ub, params_carCuboid.sz_lb ...
                                               );  
                                           
            bvolumeQ1 = bvolumeQ1 - repmat([0 params_carCuboid.avg_Sz(2)/2 0], size(bvolumeQ1,1), 1);  
        else
           
            B1Q = detectionsQ{i}.origin;
            bvolumeQ1 = detectionsQ{i}.bvolume;
            
        end
        
        % TODO
             % propagate noise and find new noise .... for now same noise             
        %                       
        
        % 车在当前帧的坐标原点
        B2Q = motion(1:3) + B1Q ;
        bvolumeQ1 = bvolumeQ1 + repmat(motion(1:3)', size(bvolumeQ1,1),1);
        % CUBOID IN TRAIN FRAME        
        
        [bvolumeQ2, k2] = getBoundingVolume(B2Q, bvolumeQ1, motion(4), ... 
                                             detectionsQ{i}.sigma_3D, ...
                                             params_carCuboid.sz_ub, params_carCuboid.sz_lb ...
                                            );                 
        
       % offset the h/2 as car ar to be on road             
        
        % 计算边界体积投影并保存为bvolume_proj变量
        bvolume_proj = (params_2D3D.K*bvolumeQ2')';
        bvolume_proj(:,1:3) = bvolume_proj(:,1:3)./repmat(bvolume_proj(:,3), 1,3);
        kbvolume_proj = convhull(bvolume_proj(:,1:2));
        bvolume_proj = bvolume_proj(kbvolume_proj,:);
        
        % 更新结构体数据
        
        detectionsQ{i}.bvolume = bvolumeQ2;             
        detectionsQ{i}.bvolume_proj = bvolume_proj;
        detectionsQ{i}.yaw  = detectionsQ{i}.yaw+motion(4); 
        detectionsQ{i}.origin  = B2Q;        
        detectionsQ{i}.k = k2;       
        
                        
    end

将结构体输入得分矩阵计算函数,获取2D-2D,3D-3D和3D-2D得分,并将2D-2D得分矩阵归一化:

[scoreMat_3D3D, scoreMat_3D2D] = generateScoreMatrices(detectionsQ, detectionsT);
            [scoreMat_2D2D]                = generate2D2DScoreMatrix(features_2D2D{i}, features_2D2D{i+1});
            
            %normalize scoreMat2D2D from -1:1 to 0:1    
            max2D2D = max(scoreMat_2D2D');
            max2D2D = repmat(max2D2D', 1, size(scoreMat_2D2D,2));
            min2D2D = min(scoreMat_2D2D');
            min2D2D = repmat(min2D2D', 1, size(scoreMat_2D2D,2));
            scoreMat_2D2D = (scoreMat_2D2D - min2D2D) ./ (max2D2D-min2D2D +0.0000001) ;
            
            scoreMat_viewpoint = zeros(size(scoreMat_3D3D)); 

其中,generateScoreMatrices函数输入前后两帧的结构体,生成3D-3D和3D-2D得分矩阵:

function [scoreMatrix_3D3D, scoreMatrix_3D2D] = generateScoreMatrices(detectionsQ, detectionsT)

    scoreMatrix_3D3D = zeros(length(detectionsQ), length(detectionsT));
    scoreMatrix_3D2D = zeros(length(detectionsQ), length(detectionsT));
    
    
    % 3D 3D 得分   
    for i = 1:length(detectionsQ)                
        for j = 1:length(detectionsT)
            scoreMatrix_3D3D(i,j) = get3D3DMatchScore(detectionsT{j}.bvolume, detectionsQ{i}.bvolume);            
            scoreMatrix_3D2D(i,j) = get3D2DMatchScore(detectionsT{j}.bbox, detectionsQ{i}.bvolume_proj);            
        end
    end
    
end

其中,get3D3DMatchScore函数:

function score = get3D3DMatchScore(bvolumeQ, bvolumeT)

    % 获取物体底部位置; 接着计算3D-3D得分   
    kbvolumeQ_xz = convhull(bvolumeQ(:,1),  bvolumeQ(:,3));%计算物体凸包
    bvolumeQ_xz  = [bvolumeQ(kbvolumeQ_xz,1) bvolumeQ(kbvolumeQ_xz,3)];
        
    kbvolumeT_xz = convhull(bvolumeT(:,1),  bvolumeT(:,3));
    bvolumeT_xz = [bvolumeT(kbvolumeT_xz,1) bvolumeT(kbvolumeT_xz,3)];
    
    % 比较车辆位置和搜索区域交集
    [x_in, y_in] = polybool('intersection', bvolumeQ_xz(:,1), bvolumeQ_xz(:,2), bvolumeT_xz(:,1), bvolumeT_xz(:,2));    
    
    % 计算车辆位置在搜索区域占比    
    area_in = polyarea(x_in, y_in);    
    bvolumeQ_xz_area  = polyarea(bvolumeQ_xz(:,1), bvolumeQ_xz(:,2));    
    
    score = area_in / bvolumeQ_xz_area;    
end

get3D2DMatchScore函数:

% 计算车辆位置与搜索区域交集
function [score] = get3D2DMatchScore(bbox, searchRegion)
    
    convShapeQ = [bbox(1) bbox(2);
                  bbox(3) bbox(2);
                  bbox(3) bbox(4);
                  bbox(1) bbox(4)];
              
    convShapeT = searchRegion;
    
    % find the intersection of bbox and search region
    [x_in, y_in] = polybool('intersection', convShapeQ(:,1), convShapeQ(:,2), convShapeT(:,1), convShapeT(:,2));    
    
    % compute percentage area of bbox i.e. Q present in the search region    
    area_in = polyarea(x_in, y_in);    
    convShapeQ_area  = polyarea(convShapeQ(:,1), convShapeQ(:,2));    
    
    score = area_in / convShapeQ_area;    
    
end

将2D-2D,3D-3D和3D-2D得分加权相加后保存到scoreMat_all:

 if(~isempty(scoreMat_2D2D))                           
                
                scoreMat_3D2D_mask = (scoreMat_3D2D > 0);
                scoreMat_2D2D = scoreMat_2D2D .* scoreMat_3D2D_mask;
                
                scoreMat_all =  c33*scoreMat_3D3D + c32*scoreMat_3D2D + c22*scoreMat_2D2D + cvp*scoreMat_viewpoint;
                
            else
                
                scoreMat_all = c33*scoreMat_3D3D + c32*scoreMat_3D2D + cvp*scoreMat_viewpoint;
            end  

使用匈牙利算法计算两个物体间的联系:

if(hungarian_association)
                % 使用Hungarian算法找循两个物体间的联系
                [assign, cost]=  munkres(-scoreMat_all);
                
                % give IDs to the  Ts based on assignment or give new ID
                
                for ii = 1:length(assign)
                    
                    % ADD: scoreMat_all(ii,assign(ii)) > 0.005 , to handle
                    % death and birth issues. Not very useful though.
                    if(assign(ii) > 0 && scoreMat_all(ii,assign(ii)) > 0.0   )
                        detectionsT{assign(ii)}.dno = detectionsQ{ii}.dno;
                    end
                end
                
            end

若没有使用匈牙利算法,则通过加权得分矩阵判断前后两帧检测的物体是同一辆车:

if(~hungarian_association)
                
                for ii = 1:length(detectionsQ)
                    
                    scoreQT = scoreMat_all(ii,:);
                    [max_v, max_id] = max(scoreQT)
                    
                    % 如果最大得分超过阈值,则判定与前一帧检测出的车辆相同
                    if(max_v > 0.15)
                        % give the Q's id to matching T
                        detectionsT{max_id}.dno = detectionsQ{ii}.dno;
                        
                    end
                    
                end
            end

若当前帧的车辆未在下一帧找到匹配车辆,车辆总数加1:

 for ii = 1:length(detectionsT)
                
                if(detectionsT{ii}.dno == -1)
                    detectionsT{ii}.dno = global_id;
                    global_id = global_id + 1;
                end

将下一帧结构体赋值给当前帧,继续循环:

            first_time = 0;
            tempDQ = detectionsQ;
            detectionsQ = detectionsT;    

 

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/qq_36688879/article/details/89527939

智能推荐

文件上传漏洞利用介绍_星落.的博客-程序员宝宝

绕过js验证js验证代码分析burpsuit剔除js验证对于js前端验证,直接删除js代码之后就可以绕过js验证绕过MIME-TYPE验证验证MEMI-TYPE代码分析查看源代码分析,使用$_FILE["upload_file"]["type"]获取上传文件的MIME-TYPE类型。其中upload_file是在表单中定义的。burpsuit绕过MIME-TYPE验证利用burpsuit工具截断http请求,在repeater重放修改MIME-TYPE类型绕过验证

J2ME基本概念_丁丁宝宝的博客-程序员宝宝

本文主要介绍J2me的基本概念。因为注意到国内的java站点介绍J2me的还比较少,所以就写了本文。如果有什么不妥的地方,欢迎您email我,讨论商榷。如您要转贴,请保留原出处,并勿做删改。谢谢。1.名词这里我列出一些常见的关键词,方便与下文参照.具体解释见后文.J2ME(Java2 Platform, Micro Edition) CLDC(Connected, Limited Device

PDP Context_weixin_34402408的博客-程序员宝宝

  1)PDP(Packet Data Protocol) context   即PDP上下文,保存用户面进行隧道转发的所有信息,包括RNC/GGSN的用户面IP地址、隧道标识和QoS等。   2)SM通过PDP context的激活、修改、去激活信令流程实现会话管理。PDP context 激活流程用于建立用户面的分组传输路由;PDP context修改流程修改激活的PDP context的Q...

python【Matlibplot绘图库】多图合并显示(真の能看懂~!)_李响Superb的博客-程序员宝宝

文章目录1 Subplot多合一显示2 Subplot分格显示2.1 Subplot2.2 gridspec2.3 subplots3 图中图4 次坐标轴1 Subplot多合一显示import matplotlib.pyplot as pltplt.figure()plt.subplot(2,2,1)plt.plot([0,1],[0,1])plt.subplot(2,2,2)...

Asp Net Core Fro MongoDB Part1_weixin_30328063的博客-程序员宝宝

"没有时间了!"本次学习mongoDB,参考了如下文章: Using MongoDB with ASP.NET Core – Part II (Implementation) 来自 &lt;https://www.janaks.com.np/using-mongodb-with-aspnet-core-ii/&gt; MongoDB学习笔记(二) ...

python学来干什么-学python出来到底能干嘛_编程大乐趣的博客-程序员宝宝

在it大火的今天,越来越多的人们加入到了培训it技能的大军中来,其中python是里边新且热门的技能之一,你了解过python培训嘛?python培训出来到底能够干嘛?python会不会和其它语言一样那么难?让小编来告诉你详细信息吧!python难学嘛?python其实并不难学,对于初学者和完成普通任务,Python语言是非常简单易用的,虽然python是新语言但是他并不表示他就比C语言要难,比j...

随便推点

机器学习之基于文本内容的垃圾短信识别_小嗯子的博客-程序员宝宝_在train和test上提取的feature维度不同,那么怎么让它们相同呢

机器学习之基于文本内容的垃圾短信识别1.背景与目标2.数据探索3.数据预处理4.文本的向量表示5.模型训练与评价1.背景与目标我国目前的垃圾短信现状:垃圾短信黑色利益链缺乏法律保护短信类型日益多变案例目标:垃圾短信识别。基于短信文本内容,建立识别模型,准确地识别出垃圾短信,以解决垃圾短信过滤问题2.数据探索总体流程数据展示*观察数据,请思考:建模前需...

android绘制自定义view时,控件大小的测量与确定问题_沧海龙腾LV的博客-程序员宝宝

最近项目需求,要做以下gif效果:方法1:使用开源得gif控件,加载gif图片方法2:view平移动画方法3:继承view,activity中线程刷新方法4:继承LinearLayout,添加子控件,activity中线程刷新由于没有合适的gif图片,又不想在activity中书写太多的动画逻辑,考虑采用采用方法3和方法4封装控件的方法。本人采用的方法4,感觉比较简单,不用计算绘制onDraw,直...

深度学习-常用激活函数详细介绍_AI研习图书馆的博客-程序员宝宝_深度学习常用激活函数

本文主要介绍深度学习常用激活函数,介绍了sigmoid激活函数,tanh激活函数,relu激活函数等

多传感器融合课程笔记------信息融合_Tianchao龙虾的博客-程序员宝宝

多传感器融合之信息融合一、多传感器融合的优点冗余性:多个传感器数据对目标的描述表示是相同的(如激光雷达检测的目标信息和摄像头目标信息大体一致,允许有一定的属性误差);互补性:由于每个传感器的探测范围和探测属性不同,传感器之间的目标属性可以相互补充( 如前方障碍物的颜色信息, 激光雷达识别不出来);合作性:不同传感器在处理信息时会对其他信息有所依赖(如相机在建立位姿关系时,需要依赖IMU的数据);信息分层:可以根据数据融合所呈现的位置,进行分层,包括原始数据层(简称像元层)、特征层、 决策层(也叫

递归解决最长公共子序列问题(LCS)_zzzmj_的博客-程序员宝宝_递归求最长公共子序列

 前言写在前面,这篇文章是我大一的时候写的,质量比较差,没想到还是被挺多人看到了,有上千的阅读量,说实话有点内疚~~,因为之前写的太差了,看了我文章可能对这个问题更不清楚了。抱着不能坑别人的想法,我重新编辑了一下文章,希望还是能帮到大家的。其实用递归解决最长公共子序列是不好的,网上最多的方法应该是用动态规划解这个问题,动态规划解这个问题确实优秀很多,但动态规划理解起来是有难度的,我建...

VUE表格编辑单元格_Winhye的博客-程序员宝宝_vue表格编辑

1、需求:点击操作栏的“修改按钮”,使占用金额栏下单元格进入可编辑状态,并进行金额校验。2、页面表格以及欲编辑单元格增加如下代码。3、页面内操作栏增加修改编辑按钮。4、页面data中增加如下变量。5、方法中增加如下。...

推荐文章

热门文章

相关标签