I'm working with the new face tracking SDK of Kinect (Microsoft Official), and I noticed that there's difference in detection between c++ and c#-wpf example: the first one is way faster in recognition than the second (the one I want to use, actually). In the c++ version the face tracking is almost on the fly, while in the wpf one it starts ONLY when I put my entire body (so the entire skeleton) in the FOV of Kinect.
Did anyone found out why? I noticed that the skeletonframe provided shows the property "Trackingmode = default", even though I set the kinect skeleton stream on seated.
colorImageFrame.CopyPixelDataTo(this.colorImage);
depthImageFrame.CopyPixelDataTo(this.depthImage);
skeletonFrame.CopySkeletonDataTo(this.skeletonData);
// Update the list of trackers and the trackers with the current frame information
foreach (Skeleton skeleton in this.skeletonData)
{
if (skeleton.TrackingState == SkeletonTrackingState.Tracked
|| skeleton.TrackingState == SkeletonTrackingState.PositionOnly)
{
// We want keep a record of any skeleton, tracked or untracked.
if (!this.trackedSkeletons.ContainsKey(skeleton.TrackingId))
{
this.trackedSkeletons.Add(skeleton.TrackingId, new SkeletonFaceTracker());
}
// Give each tracker the upated frame.
SkeletonFaceTracker skeletonFaceTracker;
if (this.trackedSkeletons.TryGetValue(skeleton.TrackingId,
out skeletonFaceTracker))
{
skeletonFaceTracker.OnFrameReady(this.Kinect,
colorImageFormat,
colorImage,
depthImageFormat,
depthImage,
skeleton);
skeletonFaceTracker.LastTrackedFrame = skeletonFrame.FrameNumber;
}
}
}
The code is the one provide my microsoft with the 1.5 SDK.
I had some information in other forums, specifically here (Thanks to this guy (blog)):
MSDN forum link
Basically, in the c++ example all the methods to track the face are used, both color+depth and color+depth+skeleton, while in the c# only the latter is used. So it only starts when you stand up.
I did some tests, but the other method is still not working for me, I did some modification to the code but with no luck. Here is my modification:
internal void OnFrameReady(KinectSensor kinectSensor, ColorImageFormat colorImageFormat, byte[] colorImage, DepthImageFormat depthImageFormat, short[] depthImage)
{
if (this.faceTracker == null)
{
try
{
this.faceTracker = new Microsoft.Kinect.Toolkit.FaceTracking.FaceTracker(kinectSensor);
}
catch (InvalidOperationException)
{
// During some shutdown scenarios the FaceTracker
// is unable to be instantiated. Catch that exception
// and don't track a face.
//Debug.WriteLine("AllFramesReady - creating a new FaceTracker threw an InvalidOperationException");
this.faceTracker = null;
}
}
if (this.faceTracker != null)
{
FaceTrackFrame frame = this.faceTracker.Track(
colorImageFormat,
colorImage,
depthImageFormat,
depthImage,
Microsoft.Kinect.Toolkit.FaceTracking.Rect.Empty);
//new Microsoft.Kinect.Toolkit.FaceTracking.Rect(100,100,500,400));
this.lastFaceTrackSucceeded = frame.TrackSuccessful;
if (this.lastFaceTrackSucceeded)
{
if (faceTriangles == null)
{
// only need to get this once. It doesn't change.
faceTriangles = frame.GetTriangles();
}
this.facePointsProjected = frame.GetProjected3DShape();
this.rotationVector = frame.Rotation;
this.translationVector = frame.Translation;
this.faceRect = frame.FaceRect;
this.facepoints3D = frame.Get3DShape();
}
}
}
frame.TrackSuccessful is always false. Any idea?
I finally figured it out and made a post on MSDN forums regarding what else needs to be done to get this working.
It's here.
Hope that helps!
Related
The function (code below) simply moves a character node (contact.nodeA) back after it has penetrated a collision node (contact.nodeB) that it's not allowed to pass through.
In a scenario where a user is trying to move the character node (contact.nodeA) while it's stuck in a corner, etc. the scene/character repeatedly shakes/glitches while its position is repeatedly being moved back away from contact.nodeB at 60hz.
Is it possible with SCNPhysicsContact in SceneKit to prevent contact.nodeA from moving forward into contact.nodeB in the first place? (ie., why does "contact.penetrationDistance" occur at all? Why can't we just be alerted when a physics contact has begun w/o contact.nodeA actually moving forward into contact.nodeB? We are, after all, catching the contact on "didBegin" yet there's already penetration that we have to readjust.)
(When a user is trying to move the character node while it's "stuck" in a corner, or between two rocks, etc.)
Or, is there another solution to prevent the repeated shaking of contact.nodeA as its position is repeatedly being readjusted to compensate for its penetration into contact.nodeB? Transform instead of position change?
I've already tried setting momentum to 0.0 and removing all animations, etc. However, since contact is still occurring with contact.nodeB, contact.nodeA cannot be moved away since its momentum is still being stopped during contact with nodeB.
Any help would be greatly appreciated.
private func characterNode(_ characterNode: SCNNode, hitWall wall: SCNNode, withContact contact: SCNPhysicsContact) {
if characterNode.name != "collider" { return }
if maxPenetrationDistance > contact.penetrationDistance { return }
var characterPosition = float3(characterNode.parent!.position)
var positionOffset = float3(contact.contactNormal) * Float(contact.penetrationDistance)
maxPenetrationDistance = contact.penetrationDistance
characterPosition.y += positionOffset.y
replacementPositions[characterNode.parent!] = SCNVector3(characterPosition)
}
extension GameViewController : SCNPhysicsContactDelegate {
func physicsWorld(_ world: SCNPhysicsWorld, didBegin contact: SCNPhysicsContact) {
if gameState != .playing { return }
contact.match(BitmaskWall) {
(matching, other) in
if other.name == "collider" {
self.characterNode(other, hitWall: matching, withContact: contact)
}
}
}
Last year was my first successful year in teaching my students to create catch and avoid games in Flash with AS3. This year is getting better. Each year I come here for help at least once.
I'm trying to add shooting into the possibilities for their second game project. I can make the shot happen from the ship, gun, whatever, and make it move, and get rid of it when it is off screen, but have not figured out a clean way to have both the shot and the target go away (removeChild and array.splice) upon collision.
The code I have sort of works, but I keep getting the error,
TypeError: Error #1010: A term is undefined and has no properties. at
DropShootV02_fla::MainTimeline/checkShots()
.
Normally I know that this is because of a mismatch between objects and index numbers, but this is related to the call to a second array in removing boxes and bullets.
For simplicity I'll just include the shooting code. Similar organization creates and drops the boxes.
Any help is appreciated. BTW we are not using external script in an AS file.
var shotSpeed = 18;
var shots:Array = new Array();
import flash.events.MouseEvent;
import flash.events.Event;
stage.addEventListener(MouseEvent.CLICK, fireLaser);
function fireLaser(e:MouseEvent):void
{
if (gameOn==true)
{
var shot:Shot=new Shot();
addChild(shot);
shots.push(shot);
shot.gotoAndStop(1);
shot.x = user.x;
shot.y = user.y;
trace(shots.length);
}
}
addEventListener(Event.ENTER_FRAME, moveShot);
function moveShot(e:Event):void
{
for (var i:int=shots.length-1; i>=0; i--)
{
shots[i].y -= shotSpeed;
if (shots[i].y < -25)
{
removeChild(shots[i]);
shots.splice(i,1);
}
}
}
addEventListener(Event.ENTER_FRAME, checkShots);
function checkShots(e:Event):void
{
for (var i:int=shots.length-1; i>=0; i--)
{
for (var k:int=boxes.length-1; k>=0; k--)
{
if (shots[i].hitTestObject(boxes[k]))
{
if (boxes[i].type == "good")
{
score--;
scoreDisplay.text = "Score:" + score;
}
if (boxes[i].type == "bad")
{
score++;
}
removeChild(boxes[k]);
boxes.splice(k,1);
//if I leave this part out I get no errors,
//but but the bullet goes on to strike again
removeChild(shots[i]);
shots.splice(i,1);
}
}
}
}
Thanks kaarto:
I had tried that previously and kept getting the same error. I used that elsewhere in this game code. Turns out I needed to moderate how often the player was shooting. I changed from shooting with the mouse to using space instead and now the problem is gone. Break is definitely a good one here.
Merge move shot and checkShots in one ENTER_FRAME handler.
I've been on and off creating a text adventure using a rudimentary engine I made for about a year.
For the story, I have a object array(is that what it's called?) with various story data stuff that I parse through
I've been told that using it the way I am is stupid because it's supposed to be used for other stuff but I only use it because it was easy to learn how to parse the data since I was a beginner
It's getting tedious writing the story and doing stuff for each part (creating the backgrounds and such) since it's so long.
Is there any kind of way I can make it easier for me to write the story?
Here's the object array with a single part set up (with choices)
public static var parts:Object =
{
"0":
{
"text":"Text here",
"choices":
{
"response1":
{
"text":"Response1",
"nextPart":"1"
},
"response2":
{
"text":"Response2",
"nextPart":"2"
}
},
"background": Assets.AssetClass.Background1,
"BGM":"bg1"
},
}
Here's an example of how my engine deals with parts and changing them:
I have a input checker to check when enter is pressed and then do stuff depending on what is on the screen
public function onEnter(button:KeyboardEvent):void
{
if (button.keyCode == 32 && !Won)
{
if (Dead && textFinished && !choosing) // pressing enter while dead and the text is still writing
{
curPart = parts[curPart]["lastPart"] // lastPart will only be stored in parts that have the player die
textFinished = false
Dead = false;
myTextField.text = ""
counter = 0;
sInt = setInterval(addCharackter, textSpeed)
if (stage.getChildByName("cText"))
{
stage.removeChild(continueText)
}
if (parts[curPart].hasOwnProperty("background")) //check if the background needs to change.
{
if (stage.getChildByName("img"))
{
stage.removeChild(background)
}
background = new Background(parts[curPart], 800, 600)
stage.addChildAt(background, 0)
}
}
if (!textFinished && !choosing)// pressing enter when there's no choices on the screen and the text isn't finished and the text is still writing
{
this.myTextField.text = this.parts[this.curPart]["text"];
clearInterval(this.sInt);
this.textFinished = true;
if (parts[curPart].hasOwnProperty("choices"))
{
choosing = true
createOptions(); // function for parsing through the responses bit of that part and displaying them
}
else
{
stage.addChildAt(continueText, 2)
}
if (parts[curPart].hasOwnProperty("lastPart"))
{
Dead = true;
dead()
}
}
else if (textFinished && !choosing && !Dead) // pressing enter if there's no choices on the screen and there's no choices (it'll take you to the next part)
{
trace("Text finished!!")
curPart = parts[curPart]["nextPart"]
myTextField.text = ""
counter = 0;
sInt = setInterval(addCharackter, textSpeed)
textFinished = false;
if (parts[curPart].hasOwnProperty("background"))
{
if (stage.getChildByName("img"))
{
trace("Removed!")
stage.removeChild(background)
}
background = new Background(parts[curPart], 800, 600)
stage.addChildAt(background, 0)
}
if (parts[curPart].hasOwnProperty("BGM")) // does the current part have a new background music?
{
trace("Music!!!!")
sndBGMusic = musicArray[parts[curPart]["BGM"]]
sndBGMusicChannel.stop()
sndBGMusicChannel = sndBGMusic.play(0, 9999999999)
stage.addChildAt(background, 0)
}
stage.removeChild(continueText)
}
}
}
A couple ideas here. These are just things I would do differently than what you have done. I don't guarantee that they are better in any way, but see what you think.
I would have a global variable for _curPart. And I would have a custom class called Part. That class would have properties like _BGM, _bgImage etc. It could have a _choicesArray as a property as well. I'd have other global variables like _hasCandle. Or you can store items in an array and if you need the candle just check if candle is in the array. These global variables will persist from one part to the next.
Then you can access the properties of the part you are in by doing _curPart._bgImage. To me, that looks and feels cleaner.
And to create a new part it could look like (incomplete):
var p15:Part = new Part();
p15._bgImage = image15;
p15._BGM = song10;
//...
The last thing I'd recommend is to try to refactor where you can. For example, where you have //pressing enter if there's no choic... replace all of that code in that bracket with a one or a few function calls (whatever makes the most sense and allows you to reuse code). It just makes it easier to see what's going on, I think. So instead of all these if blocks, just a function like nextPart(); and then that function will have all your if blocks in it. Make sense? Personal preference, but when things are getting complicated, refactoring helps me clear out the cobwebs. Just like you do with dead() and createOptions() but I'd just take it one step further. This won't make your code more efficient, but it might make writing your code more efficient which is paramount in my book (until it's not).
I am using android studio's example bluetoothlegatt, which has a read function and and works completely find (it interacts with the TI keyfob I have which sends over values when pressed).
However, I have no idea how to put in writeCharecteristic because my previous attempts have failed.
Can someone help me with this? I would like to send a "1" over so that the keyfob's led lights up.
Edit: I have tried something like this
public void writeCharacteristic(byte[] value) { BluetoothGattService LumService = mBluetoothGatt.getService(UUID_LUM_APP);
if (LumService == null) {
System.out.println("LumService null"); return; }
BluetoothGattCharacteristic LumChar = LumService.getCharacteristic(UUID_LUM_CHAR);
if (LumChar == null) {
System.out.println("LumChar null"); return; }
LumChar.setValue(value); boolean status = mBluetoothGatt.writeCharacteristic(LumChar);
System.out.println("Write Status: " + status);
This is not my code, though, just an example. I want to integrate this into the bluetoothlegatt given already, but I have had no success with it.
I was working on writing a screenshot thing, and found this excellent topic for Mac: How can I get screenshot from all displays on MAC?
I was wondering if anyone has the equivalent for x11 library? To get all the monitors and then screenshot them all?
I had found this topic: https://stackoverflow.com/a/5293559/1828637
But the code linked from there is not as easy to follow for a novice like me.
Will RootWindow(3) get the area of all the monitors combined? Then I can go through and get the monitors dimensions then XGetImage those sections on the return of RootWindow?
I had come across this topic: How do take a screenshot correctly with xlib? But I'm not sure if it has multi-monitor support. I do this in ctypes so I cant test that code easily without going through the grueling task of writing it first. So I was wondering if this is correct or how would I modify it to handle multi mon please?
Edit
The poster there shared his code, it is seen here: https://github.com/Lalaland/ScreenCap/blob/master/src/screenCapturerImpl.cpp#L96 but it's complicated and I don't understand it. It uses functions like XFixesGetCursorImage which I can't find in the documentation, and I don't see how the multi monitors work there. Author of that topic warned he doesn't remember the code and it may not work with modern Linux.
This is not a perfect answer to the question, but the following code could be modified to get a very fast version of your desired end result:
https://github.com/Clodo76/vr-desktop-mirror/blob/master/DesktopCapture/main.cpp
The DesktopCapturePlugin_Initialize method converts all the displays into objects:
UNITY_INTERFACE_EXPORT void UNITY_INTERFACE_API DesktopCapturePlugin_Initialize()
{
DesksClean();
g_needReinit = 0;
IDXGIFactory1* factory;
CreateDXGIFactory1(__uuidof(IDXGIFactory1), reinterpret_cast<void**>(&factory));
IDXGIAdapter1* adapter;
for (int i = 0; (factory->EnumAdapters1(i, &adapter) != DXGI_ERROR_NOT_FOUND); ++i)
{
IDXGIOutput* output;
for (int j = 0; (adapter->EnumOutputs(j, &output) != DXGI_ERROR_NOT_FOUND); j++)
{
DXGI_OUTPUT_DESC outputDesc;
output->GetDesc(&outputDesc);
MONITORINFOEX monitorInfo;
monitorInfo.cbSize = sizeof(MONITORINFOEX);
GetMonitorInfo(outputDesc.Monitor, &monitorInfo);
// Maybe in future add a function to identify the primary monitor.
//if (monitorInfo.dwFlags == MONITORINFOF_PRIMARY)
{
int iDesk = DeskAdd();
g_desks[iDesk].g_width = monitorInfo.rcMonitor.right - monitorInfo.rcMonitor.left;
g_desks[iDesk].g_height = monitorInfo.rcMonitor.bottom - monitorInfo.rcMonitor.top;
auto device = g_unity->Get<IUnityGraphicsD3D11>()->GetDevice();
IDXGIOutput1* output1;
output1 = reinterpret_cast<IDXGIOutput1*>(output);
output1->DuplicateOutput(device, &g_desks[iDesk].g_deskDupl);
}
output->Release();
}
adapter->Release();
}
factory->Release();
}
Then the OnRenderEvent method copies a frame from the display into a texture (provided by unity in this case):
void UNITY_INTERFACE_API OnRenderEvent(int eventId)
{
for (int iDesk = 0; iDesk < g_nDesks; iDesk++)
{
if (g_desks[iDesk].g_deskDupl == nullptr || g_desks[iDesk].g_texture == nullptr)
{
g_needReinit++;
return;
}
IDXGIResource* resource = nullptr;
const UINT timeout = 0; // ms
HRESULT resultAcquire = g_desks[iDesk].g_deskDupl->AcquireNextFrame(timeout, &g_desks[iDesk].g_frameInfo, &resource);
if (resultAcquire != S_OK)
{
g_needReinit++;
return;
}
g_desks[iDesk].g_isPointerVisible = (g_desks[iDesk].g_frameInfo.PointerPosition.Visible == TRUE);
g_desks[iDesk].g_pointerX = g_desks[iDesk].g_frameInfo.PointerPosition.Position.x;
g_desks[iDesk].g_pointerY = g_desks[iDesk].g_frameInfo.PointerPosition.Position.y;
ID3D11Texture2D* texture;
HRESULT resultQuery = resource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void**>(&texture));
resource->Release();
if (resultQuery != S_OK)
{
g_needReinit++;
return;
}
ID3D11DeviceContext* context;
auto device = g_unity->Get<IUnityGraphicsD3D11>()->GetDevice();
device->GetImmediateContext(&context);
context->CopyResource(g_desks[iDesk].g_texture, texture);
g_desks[iDesk].g_deskDupl->ReleaseFrame();
}
g_needReinit = 0;
}