Wavelet/Geometry Processing ELEC484 Project Proposal

So, I just sent this off:

Hi Michael,

Here's what I've got so far for my term project. I'd like to write a program which constructs 3d models of objects placed in front of my Kinect RGBD(Depth) camera. 

1. read an RGBD (RGB and depth) point cloud from my Kinect (OpenNI or libfreenect)
2. convert that point cloud into a mesh object (PCL or VCG)
3. implement a few subdivision schemes (Butterfly or Kobbelt √3?) on the resulting mesh object (straight C++)
4. import the resulting subdivided mesh objects into a viewer program (Meshlab or Blender) 
5. Optional goals: 
1. implement a point cloud to mesh object converter in C++ using isosurface extraction or front meshing algorithms to replace PCL/VCG from #2
2. paint the final mesh object using a Blender python script and the RGB data from the initial point cloud in #1
3. implement an un-subdividing scheme to generate lower polygon count models
4. put those models in a dorky little game using Blender (I've used the Blender game engine before, a little, and there are many Free sample games available to work from)

I plan to spend the next couple of weeks learning as much as I can about all of the tools mentioned below, in hopes of using them in my elec 484 and elec 486 projects, and just for fun. 

My proposed schedule is:

1. Spend the next two weeks learning as much as I can about all of the tools mentioned below, in hopes of using them in my elec 484 and elec 486 projects, and just for fun. Have a working prototype of this project running which does not involve me implementing any course related algorithms (Blender already supports Linear and Catmull-Clark subdivision. If my implementations are even remotely efficient I'll be submitting them to the Blender project :) )

2. If I've succeeded, spend the next month implementing Butterfly and Kobbelt √3 subdivision, and after that as many secondary goals as time allows

3. If I've failed to get a mostly working prototype in the first two weeks, I'll abandon the point cloud to mesh object conversion until the subdivision schemes are working properly.

Terms:

• OpenNI Natural Interaction API (includes Kinect plugin & library) http://www.openni.org/
• freenect (Kinect plugin & library) http://openkinect.org/wiki/Main_Page
• KinectRgbDemo: free Kinect freenect.lib demo which saves Kinect data as point clouds or as mesh surfaces (sample mesh surface screenshot attached)  http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV5
• PCL (Point Cloud Library) – includes acquisition, filtering, feature estimation, surface reconstruction, registration, model fitting, and segmentation http://www.ros.org/wiki/pcl
• VCGLib: supports isosurface extraction and advancing front meshing algorithms http://vcg.sourceforge.net/
• Blender (3D modelling / game engine / content creation / etc toolkit which I used a fair bit in high school) http://blender.org
• Blender import scripts (.ply supported) http://wiki.blender.org/index.php/Extensions:2.5/Py/Scripts

Attachment: (image screenscapped from youtube user MontyVaca's video)

p.s. I submitted my request for patient information form today! I'm gonna have a CD of xray images etc. that I have full legal ownership of for use in projects. I'm excited :)

Kinect Goals for the summer

This semester I intend to use a Microsoft Kinect sensor platform, the OpenCV library, the libfreenect driver, and C++ to:

1. Read Kinect  RGB and Depth camera images into OpenCV (ref)
2. Convert those to RGBD point clouds
3. Convert those into coloured 3d mesh surfaces (see PCL Point Cloud Library, http://www.pointclouds.org/documentation/tutorials/,  http://www.cs.berkeley.edu/~jrs/meshs08/, http://www.cfd-online.com/Wiki/Mesh_generation, )
4. Optimize those using Linear, Butterfly, and Kobbelt √3 subdivision schemes (see Elec 486 course pack, and possibly 1, 2 )
5. Write a 3-dimensional audio reverberation/echo simulator (see 1, 2, 3)
• which takes as inputs:
1. a set of audio recordings of songs and of individual notes
2. a set of x,y,z locations for these recordings to emanate from
3. an event notification whenever the user moves their hand over a (virtual) audio source location, so that the audio can be paused/resumed or modified to give the effect of virtual reality musical instruments
6. Document everything, and submit steps 3 and 4 for inclusion in the OpenCV library which currently has those functions sitting on its to-do list.

On a semi-related note, about a year and a half ago I was looking at contributing to the Blender project. I didn't contribute anything, but I did study the code base a bit and I think I might be able to handle importing the mesh object and a skeleton (as provided by OpenCV or OpenNI) into blender for animation purposes. I'm not sure how I would select points to attach the mesh to the skeleton, however. Inspired by 1, 2.

http://kinect.dashhacks.com/kinect-guides/2011/02/16/import-kinect-data-blender-video-tutorial

Kinect purchased

I went out and bought a Kinect, so I can do testing at home. It looks like this:

Mine is in one piece though, which is nice. Thanks IFixit, for the educational tear-down

Here's a patent explaining depth map function: http://ip.com/patapp/US2010011812.

Important links for later today:
http://www.openni.org/downloadfiles
http://openkinect.org/wiki/Getting_Started#Windows
http://opencv.willowgarage.com/wiki/InstallGuide

Ubuntu 11.04 UVic VPN Setup

I needed to setup a VPN connection to my school today so I could read the full text of some academic papers. speifically Generalized Daubechies Wavelet Families) (see here for a less technical explanation of Daubechies Wavelets)

The UVic helpdesk explains how to use the VPN, but only on Windows and MacOSX. What is this, the 90s? They really could have made this easier. It's not like there's a shortage of linux users on campus. Heck, UVic has about 100 linux computers available to students, and that's just the ones in labs I know about.

These instructions work fine if you are willing to connect and disconnect from the VPN manually each time, using the command line. You'll also need the .pcf file provided by the helpdesk. Scroll down to find the right link for cisco-decrypt.c.

I am running Ubuntu Linux 11.04, but these instructions should work for most Linux distributions with minimal modification. These instructions use the command line, and there's gotta be a more user friendly way to do this, let me know if you think of one.

Update: 
dmg (2) sent me this helpful note on how to use VPNC instead of the command line:

thanks for the info on how to get VPN working. 

It still works (Ubuntu 12.04 LTS)  

After following your instructions, all that is needed is to install the VPN software (vpnc) and have a valid configuration file (see attached). This file can be distributed since the user/password section is not filled.

thanks again,
--dmg 

The VPNC network manager has ubuntu packages named "vpnc" or "network-manager-vpnc".

He also attached his config file, uvic.conf, contents:

## generated by pcf2vpncIPSec ID EveryoneIPSec gateway vpn1.uvic.caIPSec secret Anything
IKE Authmode psk
## To add your username and password,## use the following lines:# Xauth username <your username># Xauth password <your password>

Thanks dmg! You make the internet a better place!

Dynamic Templates

I hadn't used blogspot for the longest time until the other day when I started this blog, and I just stumbled into Dynamic Templates. If you go here: http://murraymeehan.blogspot.com/view/ you'll see what I mean, it's a set of alternative page layouts which are way, way nicer (in my opinion).

There is officially no way to set these dynamic templates as the default template for your blog yet.

Back to Classes

I returned to class at the beginning of last week, as an electrical engineering student, and have now begun working on my term projects. I'm really excited about them, so here's a rough idea of what I'm thinking for posterity's sake:

For my wavelet/multirate/geometric processing class, I'll be making a computer game using an Xbox 360 Kinect sensor platform and the Computational Geometry Algorithms Library (CGAL). I already checked, and Tetris has been done so that's me back to the drawing board :{)

For my audio signal processing class I will be making a Voice Encoding (Vocoder) Android app using Java for the user interface, C++ for the intensive audio processing, and JNI to communicate in between. The professor doesn't allow creative control over anything but the target platform for this one, so I'll have to pour my leftover creativity into a heart rate monitor or a game of some kind.

For my medical image processing class I will be making a computer vision based medical diagnosis program, and right now I think I want to use ultrasound imaging but the professor will unveil a medical image database for us to work with next week(ish) so it depends on what she's got in store for us.

I've been reading and planning for these projects for a week and a half now and I'm more excited about them every day!

Some folks might notice a trend in my class names, and yes I'm basically doing a degree on digital signal processing. It's great :D

Soldering is relaxing, and I hope it's not because of the fumes.

This evening I got the urge to go through my junk bin, find a circuit board, and harvest the sweet juicy components therein. It was relaxing, interesting, and it added about 55 new little bits and bobs to my collection. Special additons included a pretty blue transformer named NT0247NL and an infrared receiver/transmitter. I'm not sure if the 3rd one from the left is a combined receiver and trasmitter, or if the one on the far left is the transmitter, but it should be neat to figure it out. I've got a remote which matches the receiver around somewhere, wonder what I'll do with that.

While trying to identify NT0247NL I found the spec sheet for the DVD player these are all from. It didn't clear much up though. I think I'll disassemble another dvd player tomorrow. :)

I got a star!

This morning I woke up, put on my jeans, and found this in my pocket. A green foam star and some jelly beads, leftover from a little foam grow-your-own-dinosaur capsules. But star-shaped.

It was a nice start to the day.