%Part 6 - Close timer + video objects for a clean finish Open 'C:\Program Files (x86)\Dasher\Dasher 4.11\dasher.exe' %find the sum of intensities of the neghbourhood for each pointĬIm=getdata(DepthVid,1,'double') %convert to doubleĬIm=cobble(CIm) %reconstruct image- Matlab parses the image T4fbench=0 %holds total val of intensities captured at right point T4bbench=0 %holds total val of intensities captured at left point T4hbench=0 %holds total val of intensities captured at centre point %Part 5 - Calibration + starting video + timer objs %Part 4d - Display Crosshairs + display imagesĭIm=rcross(DIm,rows/2,(cols/2)+20) %right Mouse.mouseMove(ccur,rcur) %return to centr of crosshair If DImgs(rows/2,cols/2)=0 %if centre point not occupied Mouse.mouseMove(ccur,rcur+nopix) %move mouse S Mouse.mouseMove(ccur,rcur-nopix) %move mouse N Mouse.mouseMove(ccur-nopix,rcur) %move mouse WĮlseif Back=0 & Front>0 %if only left point occupied* Mouse.mouseMove(ccur-nopix,rcur+nopix) %move mouse SW Mouse.mouseMove(ccur-nopix,rcur-nopix) %move mouse NW Mouse.mouseMove(ccur+nopix,rcur) %Move mouse EĮlseif Horizontal0 %if only left point occupied* Mouse.mouseMove(ccur+nopix,rcur+nopix) %move mouse SE Mouse.mouseMove(ccur+nopix,rcur-nopix) %move mouse NEĮlseif Front=0 & Back>0%if only right point occupied* If Back=0 & Front>0 %if only left point occupied* If Horizontal>hbench+(hbench*0.1)%if centre point intensity Horizontal=th/9 %average of cetre point neighbourhoodīack=DImgs((rows/2),(cols/2)+20) %right pointįront=DImgs((rows/2),(cols/2)-20) %left point %Part 4c - Control mouse based on object position If obj>0%if obj>0 then there is an object in the centre %mke the central object in blue + delete non central objs Obj=objmatrix(rows/2,cols/2) %select object that occupies the =bwlabel(DImgs) %find # of objects and their %Part 4b - Finding and selecting object of interest %white or black depending n whter it is greater or less than %than 0.7 othewise the intensity is zero, DIm will either be %threshold images, DImgs will only show intensities of greater =size(DImgs) %get #rows and collumns in image Trigger(DepthVid)%trigger DepthVid to capture imageĭIm=getdata(DepthVid,1,'double') %convert to doubleĭIm=cobble(DIm) %reconstruct image- Matlab parses the imageĭImgs=rgb2gray(DIm) %create gs DImage for intensity analysis %Part 4a - Aquiring, arranging and thresholding images 'Position',[0.5 0.02 0.16 4 - Control function - this is effectively the main function Stopb=uicontrol('Parent',window,'Style','pushbutton','String'. 'Position',[0.22 0.02 0.16 stop button which calls the stopbCallback fctn Startb=uicontrol('Parent',window,'Style','pushbutton','String'. %create start button which calls the startbCallback fctn %create timer that calls the dispim fctn every 0.05 secs Triggerconfig(DepthVid, 'Manual') %trigger Depthvid manually Set(DepthVid,'TriggerRepeat',Inf) %infinite amount of triggers Set(DepthVid,'FramesPerTrigger',1) %capture 1 frame every time DepthVid %Part 2 - Set up video aquisition parametersĭepthVid = videoinput('winvideo', 2) %make object for depth camera Th=0 %will hold total for sum of neighbourhood of center point
Hbench=0 %will hold value for normal intensity at center point Rcur=425 %row where dasher's crosshair is centredĬcur=650 %column where dasher's crosshair is centred Screensize=get(0, 'screensize') %get pixel info for screenĬl=screensize(3) %number of column pixels Import %import java class to allow matlab The kinect is to the right of the user and in order to move the mouse up, down left or right the user leans forward back left or right respectively.
Finally, switching to dasher (not controlled by Matlab) sentences can be written using mouse movement. Firstly Matlab takes a real time video stream from the depth camera, then using thresholding at certain points Matlab determines which way the user wants the mouse to move, Matlab then control moves the mouse in the appropriate direction (using a java class – Matlab can’t directly control the mouse on its own). The following is a very high level explanation of what the project does.
This uses the last post’s GUI code as a base template. This post will cover a first draft of the project which was made just to provide a seed (albeit a very crude hatchet job of a seed) from which the project can grow and be refined using more intelligent techniques and algorithms. The aim of this next project is to provide a means of communication for people with severe disabilities that cause them to neither be able to speak nor use fine motor skills.