Well This is Interesting

Now I am trying to get two windows to appear on a page.... I got the windows to appear but.. one window has all the ghost images, and one window only has the limbs - no joints. Something is clearly wrong...

Err.. 

Switched some things around in the code: 

So two containers (or really however many I make) will appear on the screen. 

Problem: It's the same motion, I need there to be different motions. Also, it takes quite a while to load... I need to find a way to have this work without the user waiting 5-10 seconds for the webpage to load. But this is because the program is doing all of the calculations before hand. I guess this is preferable to having a lagging motion. Also - the camera affects BOTH containers. I need the camera to affect each separately... much still to do. 




Like a boss: 

So there are two different motions, with two different camera controls. This took about 6-7 seconds to load but moving right along. Now to work on motion maps.

Working Ghost Images

After playing around with how to copy the skeleton completely I finally figured out a way with Three.js. Instead of writing a deep copy method I am reassigning all the values to a new object. While this could probably be more concise, it is a start at making this program full functional. I have two images with the ghost images. The first is with no opacity change, the second is with an opacity change. 

This is a skeleton at every 100 frames, Opacity 100%

Here is the same movement with an opacity change on the skeleton. Opacity 50%

While I think the opacity change is a good thing - I do believe it needs some work. As seen in the final skeleton position the overlap of the lower and full opacity skeletons just doesn't look good. This can definitely be improved on. 


Another problem I have with the ghost images is every 100 frames when the ghost image is drawn, it lags. This will be extremely annoying the user (at least it is to me) So I believe I have to find a way to make this faster so it doesn't lag, especially since there will be multiple windows on one page.

Adding Ghost Images

Trial one clearly didn't work:

Working!

So I finally got my project working :) The key to the rotation problem I had was to use quaternions. I'm thrilled I finally figured it out. Here is a video of it in the browser. I will post the live code when i stop tinkering with it.

Now the next steps I have are to put in lighting, ghost images, optimization, and run a user study! The project is moving forward.

Angles will be the death of me

So I had to basically rework my project when I found out that everything had to be generalized for any skeleton that UPenn will upload to the database server. As a result, I rewrote everything so it would follow this; however, my 'hack' from the previous post to fix the angles no longer holds valid :( 


I tried figuring out angles using only Math.atan2 and failed. Then I tried to use rotation matrices, and I still failed. Now I am trying to figure out the math using quaternions. The results I am getting from using quaternions is so close to being correct. My program feels bad for me, it really WANTS to work, but it knows it can't flat out lie to me and work when it is not supposed to :( I really hope I can figure out what is wrong by the end of today...

It really wants to work...

So Close...

I don't understand what is wrong now. The man I built is complete except for the shins... This frustrates me because it really *should* work.. It's no different that the rest of the code and yet these are the only two areas that are not lining up.. hmm.. 

Edit: 

So I still don't exactly understand everything that is going on here. But if you take out the Y rotation for the shins it seems to work perfectly... Should it? No. But it does...

This is a hack - I hope to eventually fix it. 

After dealing with the locations of the joints, I moved on to trying to get the skeleton to move. Here is a video with the skeleton moving via the root positions and rotations: 

http://www.youtube.com/watch?v=WN01W7jR7ng

I've been working on getting each joint to move, but so far the result is dreadful... 

I hate angles

Well after two days straight working on this issue I can safely say... I HATE dealing with angles. I'm getting really close to getting this correct, but I'm worried my way is sorta... hackish and won't work when I need to actually start moving my man :( 


Here is the 80% built man I have.. hopefully I can get him 100% completed by today. 

I'm not sure why his shins are wrong..

Working Through Problems

 So the BVH file gives us information as such: 



HIERARCHY
ROOT Hips
{
OFFSET 0.00 0.00 0.00
CHANNELS 6 Xposition Yposition Zposition Xrotation Yrotation Zrotation
JOINT LeftUpLeg
{
OFFSET 4.29448 0.00000 0.00000
CHANNELS 3 Xrotation Yrotation Zrotation
JOINT LeftLeg
{
OFFSET 0.00000 -19.20276 0.00000
CHANNELS 3 Xrotation Yrotation Zrotation
JOINT LeftFoot
{
OFFSET -0.02174 -15.68217 -0.01872
CHANNELS 3 Xrotation Yrotation Zrotation


                                                      etc...


Now, I have all of the joints nicely placed, as seen in the previous post; however, trying to connect the joints with cylinders is proving to be more difficult. Everything in this file is in a hierarchy. So the leftLeg is -19.2 in the Y direction from the LeftLegUp, so it's Y direction is technically '-19.2 while it's child has a Y direction of -15. This is a problem. Because with the normal distance formula it is saying that the position isn't actually about 15 units apart, but more like 4-5 units apart. 


What would fix the problem: Using global variables. 


Problem: Actually getting the global variables...


Without the global variables, this is what my wonderful skeleton looks like.. 

I'm so excited

So after about a 2 week hiatus because of my illness, I am back in the lab and getting a lot done! I'm super excited that things seem to be working. 


For my WebGL / Three.js BVH Player I have: 

• Made a Grid
• Working Rotations, Pan, and Zoom mouse events
• Successfully read a BVH file into my program using PHP
• This way I have all of the information I need to create the movements of the skeleton. And I have it reading only the numbers that I will use
• Begun to build the skeleton in the T Pose

While this is still in the beginning stages, everything is coming along nicely. Using Three.js made this project so much easier and I can't wait to get more completed!

TPose
Stage 1: 

Here are the Leg Joints!

Stage 2:

Joints of the full body!

So... Hey there Javascript

Programming in Javascript.. is a pain. As a crippled version of Java that tries to do 3D work, it took me a lot of time to get the hang of it. Have I fully grasped the beauty of the language? No. But that's ok. I'm really looking for ways to implement WebGL, a mix of Javascript and OpenGL ES 2.0. That brings us to another problem. OpenGL ES 2.0 is NOT the OpenGL that you would use with say, C++. It only has a quarter of the functions.


After playing around with WebGL for a few days, I realized that its almost.. silly to not use a framework. There are about 15 different ones out right now and implementing one makes coding for WebGL just so much easier. No longer do you have to worry about the silly setups and 200 lines of code to just make three squares spin (like they did in my last post). The trick is, picking the framework that will work best for your project.


At first I started with Processing.js. I figured it was popular, had a lot of documentation, and had examples as well. Their website was laid out nicely and has a fully documented API with examples (see http://processingjs.org/ for more on projecssing.js)


However, after I fiddled with it for about a week, I still could not grasp simple things. I was confused with the combination of the processing script and the javascript and how it truly interacted. Finally I got frustrated and decided to switch to a different framework: three.js.  It seems to be the most popular framwork out right now but has one huge flaw : there is very little, if any, documentation. Three.js is an extremely powerful framework, even better than processing.js (for my project at least), but one must learn by example. It is still a work in progress and keeps getting updated constantly, so hopefully there will be more documentation in the future. Three.js is a javascript file, and is easily combined with Javascript. The OpenGL ES 2.0 functions that were a hassle to deal with are implemented within Three.js and the developer no longer needs to constantly worry about the bufferset up that straight WebGL required.


After doing some research and learning more about Three.js I was able to get a working trackball camera example. Finally completing step one of my project. And it didn't even take that long.

You can rotate, pan, and zoom in the fully functional demo.

I do have to make some changes still - for one I want to zoom by spinning the scroll wheel opposed to pressing it. Also add code so this only works when pressing the "alt" key, so it is more like Maya. But I do not think these will take long. Also I want to eventually get some lighting in, because (quite contrary to what that image looks like) the circle in the middle is actually a sphere. But that is for later.


Next is to get the sphere (which I will actually make a cylinder) to move around the gird with some data points. Once that works I will work on reading in from a file - which unfortunately might be more difficult than I would have liked.

Week 3

This week has been very interesting. While it has been decently productive, I feel like I haven't accomplished much. I've been busy doing mini tutorials trying to JavaScript up and running with WebGL. I have been mildly successful, but I find that I am spending (wasting?) a lot of effort with things that shouldn't be that hard. I decided to make this process go by more quickly, to implement a frame work that has already made functions for the developer. After going through all of the many options now out there (I believe there are about 15 right now), I decided to go with processing.js. I'm fiddling around with that now. 


Also, we had a test run with capturing multiple data at once. Motion capture, bio sensors, sole pressure sensors. The process was... well there is room for improvement. But it works! We can refine the method and I'm sure it'll run a lot more smoothly!


Throughout the week I have also been re-rendering my Grand Piano and played with lighting and settings for a while. I think it looks a lot more realistic as of now! I am happy with the progress. Here are two images from the piano turntable I made: 

Contour

Full Model

So Joe gave me deadlines. The Jello Cube must be finished by Thursday. And hopefully I can get the BVH player up and running within the next few weeks. 

Week 2

So I've been doing a lot of different things in the lab this week.

• Joe showed me all of the different sensors that we will be using and showed me how to set them up / how they work.
• The list is: sole pressure sensors, force plates, bio sensors, and cameras. Eventually we will also get eye tracker sensors.
• Working on the Jello Cube project. I’m hacking away at it slowly but surely.
• Learning about WebGL. The lab wants to set up a website that holds all of our motion capture data (along with the data from the other sensors) that is readily available to the public.
  
  
  To be more specific, the website will not only have the data available for download, but it will also give a visual representation of the motions the user can download. This way when the user searches for say, a running motion, he or she doesn’t come up with 15 motions and then blindly picks which motion to download. That can be time consuming and annoying. So hopefully this website will not only have tags describing the motion, but also it will incorporate a visual representation of the motion. We are throwing around ideas such as a visual skeleton picture uploaded to the site for each motion, or a video that actually shows the motion. However we will also need a visual representation of some sort to show information from the other sensors.

To start learning about WebGL I stumbled upon this awesome website:

http://learningwebgl.com/blog/?page_id=1217

From here I am going through the lessons to learn how to set up web pages to incorporate WebGL and how to display things. Joe suggested I first draw a box, move on to drawing three boxes, then a row of boxes, rotate the boxes, then eventually replace the boxes with skeletons.

So on to what I did: to use WebGL you must have either the updated versions of Chrome, Safari, Opera, or FireFox.  It will not work on Chrome under version 10, as well as Firefox 3 and under.

Then on to actually coding, you set up a canvas, and everything will be drawn on this canvas. The program is a mixture of HTML, JavaScript, and OpenGL. Surprisingly (?), after spending a few hours finding out exactly what everything did in the sample program I looked at from the learningwebgl blog, WebGL really isn’t that hard. I’m sure that there are still a lot of things to figure out, but I’m hopeful that I’ll be able to accomplish whatever I have to implement. 

Boxes I made using WebGL

After this I went further and made the three boxes rotate. 

Rotations!

So now I will go on to trying to replace the boxes with skeletons. 

Motion Capture Review

So the first thing Joe wanted me to do when I returned was become familiar with the motion capture system. While I did work with motion capture last year.. a lot of time has gone by, so unfortunately I do not remember everything.

So the capture data that I had to take was of a UPenn student who needed data for a robot project. I was reintroduced to where the on / off switches were, where the screen buttons, the motion capture suit, and any other equipment we needed was located, and how to set up a scene so the cameras can accurately capture the data.

The 'best' part of the set up process is the calibration dance. There is a wand with markers attached to it. During this dance, we take the wand and twirl it in circles so the cameras recognize the area that they will be capturing. This process ususally takes about 5-10 minutes (depending on how accurate you want to be). The result will be - the cameras know where they are in relation to each other. However, they do not know where they are in relation to the ground. But after you reset the origin, it's good to go.

We made the student dress in an all black suit and then stuck the markers on her in their proper place.

After recording 33 motions, we had to clean the data. This part.. takes a while.

This is the first screen seen during the cleaning process. As you can see - there is no subject.

After loading the subject in, one can see the full boy skeleton of the data captured.

So this is where the 'cleaning' comes in. The blue cones represent where a marker is missing, and that is their best guess as to where the marker should be. During this section I have to fill in all of these missing spots and make sure that there are no gaps.

Here is a larger shot of a missing marker and unlabeled markers in the scene.

This could prove to be quite a frustrating process. When I was working on this, for some reason the marker for the left elbow went to the origin of the scene. I'm not sure why. I really can't begin to explain it. But it completely messes up the data for that one frame. Every time I tried to 'fix' it, it would somehow end up coming back. After spending a good hour on one marker I got frustrated and restarted the program. And guess what? I had no problems after that...

I even got to put on the suit myself because Joe needed a picture for a paper he is about to submit. While the pictures are TERRIBLE (I look dreadfully depressed), it was interesting having markers all over my face and upper body.

Oh boy..

So even with the frustrations of the silly software, it was a good way for me to get introduced to the motion capture software again!

Summer Work

So I am starting this blog, summer of 2011, to document what I learn this summer and track my progress on various projects throughout the summer. 

Things I hope to accomplish:

• Produce usable results from whatever summer project I get assigned at the lab. 
• Finish the Jell-O cube program (shouldn’t take me the whole summer) 
• Read papers and better understand work on cloth, motion capture, wacom tablet programming, and anything else Joe assigns me. 
• Update and improve my demo reel. 
• Study for GRE’s – I want to take them in July. 
• Draw, study more aspects of life drawing.  
• Fix my animation from my Spring Semester.

So hopefully I'll get done with all of this plus whatever else comes my way, while still having time to explore Philadelphia (since I didn't really do an adequate job last year) and complete any other tasks that may be of interest to me (becoming better at video games? ). Should be a busy summer!