-
Notifications
You must be signed in to change notification settings - Fork 51
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kinect 2? #80
Comments
Awesome! I just ordered mine (a little behind on the times), so won't be getting started for a few weeks. Let me know where you get to! My preference would be for it to be built in like the original Kinect. Give me a shout here if you have questions on the Source API. See openTSPS / addons / ofxTSPS / libs / ofxTSPS / include / ofxTSPS / source / Source.h and Kinect.h for reference! |
This is the addon we have been working with: On Wednesday, June 11, 2014, Brett Renfer [email protected] wrote:
|
Thanks! The integration with TSPS looks like a little more work than anticipated — I couldn't find an existing skeleton broadcaster for the Kinect V1, and most of the existing architecture seems to favor blobs over skeletons. Nothing insurmountable, but more than I had time to take on over the weekend. I've started on a basic Kinect V2 --> OSC broadcaster here: https://github.com/kitschpatrol/Kinect2StarterKit I hope to revisit TSPS integration once I've made some progress on the above. |
OK! Thanks for looking into it. I'm up in the air with the best way forward; for now we'd just use ofxKinectV2 for windows. The dream is that libfreenect2 will get to a good state soon, but we tried it this week and it's still pretty sketchy (very slow, weirdly compressed depth data). If you're just looking at broadcasting skeletons, this is something we've talked about in the past but have stuck with blob tracking to stay more general and work with non-depth cameras. It'd be great to start to think about that again! |
@kitschpatrol nice work on that Kinect2StarterKit, let me know if there's anything I can do to help or anything I can clarify. There's still work to do done on ofxKinectV2 of course so any bugs you find or anything I can do to make your life easier, I'm happy to help. |
Hi all, checking in on this thread as we hover near the 1 year mark 😄 Hey @kitschpatrol did you get further with this? I took a look at your repo and see you're sending skeletons rather than blobs, as you mentioned before. Same deal with @danthemellowman's ofxKinect2Sender. @robotconscience, I started looking through Source.h and Kinect.h in TSPS – trying to figure out the best route for adding Kinect2 compatibility. I see the virtual getDepth() method in Source.h but it doesn't seem to be used anywhere? Wondering if that's important. Looks like the core of it is implementing getPixelsRef() in the subclass (and it looks like the Kinect subclass is actually using the depth image rather than color/IR here). |
Yes! Cracking open the ancient issue! :) Matt, you're right overall. Essentially a TSPS source just needs to serve up good pixels, and for the kinect that means depth. I've started a bigger version that allows exposure of helpful params like min/max depth value.. But as you know I've been a little busy! What's the state of ofxKinectV2? I got a version working with OpenCL at a good frame rate on OS x but that's a whole other can of worms. If you're thinking windows only it might be better to use the people image instead of just depth! It's pretty good, and running open cv on that gives you nice silhouettes. End brain dump!
|
My input: MS dropped supporting/developing the Kinect Common Bridge, I'd go On Fri, Apr 24, 2015 at 2:57 PM, Brett Renfer [email protected]
joshua noble |
That's what I've been using on windows, it's a good one. I'm more curious about libfreenect2... I have a fork of Theo's addon that works with their OpenCL stuff from a ways back, but it gives the occasional nasty frame. Will have to try their newest stuff and get back here for a Mac TSPS kinect 2 strategy. |
@robotconscience Yep, was thinking windows-only for KinectV2 (with Elliot's addon). What's the "people image?" Do you mean the color image? |
The body index image. Check it out, it's sweet! Code from another project; might be shit (and I'm on my mobile so who knows how well this will paste) but worked for us! Shared/Kinect2Trigger.h #pragma once #include "Kinect2Mesh.h" class Kinect2Trigger : protected ofThread { Kinect2Trigger() : ~Kinect2Trigger(){ void setup( Kinect2Mesh & kinectRef ){ // setup body index // setup blob detector // wait for half a frame before forgetting something // trigger update loop void update( ofEventArgs & e ){ track = ofPixels( kinectManager->getKinect().getBodyIndex()->getPixelsRef()); int l_count; if ( lock() ){ if ( lastCount == 0 && l_count > 0 ){ lastCount = l_count; if ( track.size() != 0 && track.getImageType() == OF_IMAGE_GRAYSCALE ){ // debugging only ofEvent onDetect; // public props protected: void threadedFunction(){ float pixelEase = ease; for ( int i=0; i<trackBuffer.size(); i++){ trackBufferMat = ofxCv::toCv(trackAdd); bool bNeedToProcess; cv::Mat trackBufferMat; // blobs Kinect2Mesh * kinectManager;
|
Oh, the body index image! Gotcha. For this project the Kinect is mounted almost vertical so the it's a little unreliable. What would be nice though is to have a way in TSPS to pick a source (i.e. depth vs body index vs color vs IR) for inputs that support multiple image types. Looks like you've already got this "feature support" worked out for things like optical flow, skeleton, etc. so maybe that's a good place to add something like that. |
Hi @mattfelsen et al, I regret that I haven't had time to work on this any further, but glad to hear there's been some outside progress on libfreenect2 and assorted skeleton senders. |
Alright, hit a wall here trying to get @elliotwoods' ofxKinectForWindows2 addon into the TSPS project. I'm using OF 0.8.4, VS2012 Express, Windows 8.1 Pro, openTSPS/master, and ofxKinectForWindows2/0.8.4. The addon seems to work fine on its own using the example project or creating a new project from projectGenerator with & without adding the addon, i.e. the process from each of these guides: After following the second guide to get the addon into the openTSPS project, it will compile fine after adding the solution & properties files, but once I add The source changes I made are available here. I didn't commit any changes to the project file so anyone looking at this would have to add the KFW2 addon/lib stuff via one of the articles above Hopefully someone has some idea of what's going on! |
Pinging @rickbarraza as well as I just found his GitHub username 😄 |
These all look like missing declarations, are you sure your paths are all correct? I had to monkey a lot with the path settings and make sure that I rebooted post-install for things to be found correctly. You could also check all your system paths to double-check that the KINECTSDK20_DIR is correct. |
This blog post outlines all the references you need to set to have it working with OpenFrameworks (if you're doing it by hand ). http://rbarraza.com/setting-up-kinect-2-for-openframeworks/ You may need to pull the property sheet from the ofxKinectForWindows2 (it's one of the steps listed), but that's where you can find what all the dependencies seem to be for getting a c++ project referenced correctly. Let me know if that works, |
My current public tutorials have switched to using the ofxKinectForWindows2 addon for OF made by Elliot Woods et al ( and I'll have more sample lessons at github.com/rickbarraza going up this week ), but if you want to use Joshua Noble's plug-in ( which I was using for a long time too ), here is the old URL to the preview page that let you go step by step: Setup OpenFrameworks & Kinect V2 for Visual Studio Express 2012 And what you're looking for: Feel free to share the URL's. I dropped them down to preview mode since I didn't want the confusing of multiple Kinect 2 OF addons on my website right now, but those posts are still used by people who would prefer to use ofxKinect2 from Josh. |
Thanks @joshuajnoble and @rickbarraza for the tips. The guide got me up & running for a newly created project, but now I'm remembering that TSPS has a weird, non-standard OF project/directory structure that may be causing path issues. I'll dive deeper tomorrow and see if I can get it sorted out. Cheers! |
Hey @rickbarraza I'm getting 404s on the new links you just posted, though I am indeed using the newer ofxKinectForWindows2 👍 |
@mattfelsen you mean awesome, maintainable project structure? :) Shouldn't be messing you up tho. I think Joshua has the right move, def make sure the SDK dir is set. Will have time to work Thursday if you still need help then! |
I made a new project at the same level as the existing basic/multicam projects, i.e. I'm still having trouble with the openTSPS project though! All of the various settings from the .props file look like they get included/inherited/referenced properly...I also tried copying the Include/Library Directories entries that get set on ofxKinectForWindows2Lib onto the openTSPS project, but no dice. It does seem link a linker error because it seems to compile the whole project and then dump errors at the end, though maybe I'm just expecting that behavior since that's how Xcode usually presents linker errors. |
Yikes! That's a lot of errors. Can you post your project to a repo? Working on a diff project with Kinect 2 and have an idea of what it might be. |
Sure thing, just pushed here --> https://github.com/local-projects/openTSPS/commits/feature/kinect2-project A couple notes:
|
@mattfelsen et al, new working "feature-kinect2" branch for ya. I'll merge into master once I can test interoperability with ofxKinect. For now it's a separate project in standalone/basic. I also added a tweakable near/far thresh to the GUI. |
(I'll close this once a) I add in ofxKinectV2 for Mac and b) I can test if ofxKinect + ofxKinectForWindows2 play nice together) |
@robotconscience Amazing! What did you do to get it to work?? I have Kinects 1 & 2 around so I can do some basic testing. Let me know if there's anything in particular to pay attention to! |
Few things:
As far as testing goes, I would need to re-add or ofxKinect to the project and see if you can seamlessly switch between ofxKinect and the Kinect 2. I ripped it out before I figured out the Kinect.h issue, as it looked like it was giving me issues. Might be OK! |
|
Woah! ...Definitely start with a rebuild. I ran against 0.8.4 as well. Did you checkout direct into openframeworks/ ? If you're at the right level should work fine. |
Yep, project is at /c/of_v0.8.4_vs_release/openTSPS (weird, Unix-style representation of a Windows path 😄). standalone/basic/openTSPS.sln compiles fine on the master branch. I'm getting this error in both Debug & Release builds, after a Clean Solution and a Rebuild Solution. Also tried compiling OF directly from OF/libs/openFrameworksCompiled/project/vs/openFrameworksLib.vcxproj (both Debug & Release). Lots of threads on the OF forum with the same problem, all unanswered... I'm a jinx. |
Hmmmmm. Double check all of the property configs and make sure they're the same, e.g. stuff like multi-threaded DLL, visual studio toolset, optimization level, etc! My guess is it's something like that. I had to change the tool set in at least the ofxK4W project to work for VS 2k12. |
Progress! After noticing that openFrameworks was bold in the sidebar, I |
Is every window all black? Are the LEDs on in the camera? Try tweaking the near/far in the "camera" tab, and threshold in the "sensing" tab. You're close! Wish I would have remembered the startup thing, classic VS weirdness! |
Ah, had to drag the far threshold way up. Success! |
does this work? |
@elliotwoods yes-ish. Theres's an openTSPS_Kinect2 project in the feature-kinect2 branch that is working, though only supports Kinect2 (i.e. Kinect1 support was removed to get it to compile). |
@elliotwoods ditto for Matt's comments. I took the Kinect 1 stuff out mostly out of paranoia; seems to be at least compiling OK with both. Your addon is killer btw! |
@mattfelsen Support for Kinect 1 and 2 now in feature-kinect2. Merging with master once I figure out the weird OS X version. Check the Readme; it does require a proposed fix to ofxKinect to compile (looks like all newer Win ofxKinect projects need it though). |
👏👏👏 |
Hey, All the best & thanks for the great work! |
Hey @Skofo1! I was testing with Windows 8, but I don't see why it wouldn't work for you. My guess would be that the Let me know if that works, totally just a guess! The Toolset errors you're getting in VS2k10 are because the compiler is set to 2013 toolset by default, which 2010 probably doesn't recognize. |
Hey @Skofo1, just confirming that I had that branch compiling (though, an older commit facd421) on Windows 8.1, VS2012 Express, OF 0.8.4. Not sure if VS2013 changes anything for compatibility, thoughI do think that VS2013/2015 support in OF wasn't really there in v0.8.4, and has since come a long way in the upcoming 0.9. If you're trying to get it running in VS2010, here's how to change the platform toolset (look toward the bottom). I think you'd have to do this for all the projects in the solution (OF, KFW, and TSPS), which you should be able to do all at the same time (ctrl-click to select multiple projects, then hit Properties so you can edit all of them together) |
Hey, Fehler 83 error C2065: 'Kinect': nichtdeklarierter Bezeichner 140 |
note that the current but it looks like libusb errors, i.e. libfreenect not Kinect For Windows so not something I can help with |
Hey @Skofo1, it's tricky, check the Readme for a fix for ofxKinect on Windows: The problem's related to a windows-library include that conflicts between a couple of the libraries. However, it also looks like your Kinect path (from Kinect v2) is also unresolved. Have you installed the Kinect SDK? Check a few of the links above if you're having issues there! (and thanks @elliotwoods for jumping in on the KFW2 stuff!) |
Is there any update on this? I am not experienced with compiling, I need Windows binaries. Happy to help and test, I have Kinect v1 and v2, Windows and Mac available here. I am downloading 1.3.8 now for Mac to test it. |
I can confirm that on macOS Sierra 10.12.1, TSPS 1.3.8 works for Kinect v2. |
Awesome. In my experience it's still a little unstable--unfortunately haven't had time in a while to work on it! Would love some help if you're up to it! |
I would like to help, but I am not experienced in Windows or Mac development, just Unity and web. The Mac version of Kinect2 branch still works fine, no crashes after several days. Can someone make Windows binaries? |
Anyone successfully compiled this on Windows? |
Hey @innovaat, I'm planning to use TSPS with Kinect2 on Windows for a project I'm working on now, so I will probably getting a working binary together in the next week or so. I can share a download when I do! |
hey @mattfelsen, do you still have the binaries for windows ? Thanks ! |
Sorry @dagelabi , I ended up writing a different piece of software to handle the Kinect tracking for that project. It looks like the |
Is anyone working on Kinect 2 support for TSPS?
I was thinking of taking a stab at this over the weekend... either in the form of a stand-alone OSC bridge based on the OF addon @joshuajnoble has started, or by trying to integrate with TSPS.
Just wanted to check before possibly duplicating effort.
The text was updated successfully, but these errors were encountered: