I want a new thread because I like to post some codes , the little comment section wouldn't do it.
Thanks peter Deweese previously, but I still have issues..
I have a loop, getting a screen capture 30 times. Each time I got an image I call convert to append that image to video. After that, close the video file
so, anything I did wrong. on this particular one I am not getting the video
[self init_video];
for (index=0; index< 30; index++) {
[self.view.layer renderInContext:ctx];
global_image = UIGraphicsGetImageFromCurrentImageContext();
//Once I have UIImageview of global_image I call convert
[self convert];
}
[self end_video];
////////////////////////////////////////////////////////////
-(void) init_video
{
........video setup and setup adaptor
AVAssetWriterInputPixelBufferAdaptor *
avAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput sourcePixelBufferAttributes:NULL];
[videoWriter addInput:writerInput];
[videoWriter startWriting];
[videoWriter startSessionAtSourceTime:kCMTimeZero];
}
/////////////////////////////////////////////////////
-(void) convert
{..........getting the image pixel buffer
CMTime frameTime = CMTimeMake(1, 15);
CMTime lastTime=CMTimeMake(index, 15); //index is from 0 to 29 of the loop above
CMTime presentTime=CMTimeAdd(lastTime, frameTime);
//according to Peter DeWeese, needs to call twice to make it work.
[avAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:lastTime];
[avAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:presentTime];
}
//////////////////////////////////////////////////////////
-(void) end_video
{
[writerInput markAsFinished];
//want 2 seconds of video, with 15 frame for each seconds.
[videoWriter endSessionAtSourceTime:CMTimeMake(30, 15)];
[videoWriter finishWriting];
}
Related
I've written a small class that uses AVFoundation to play audio using an array. Basically an new element is appended to the array every time 'playAudio' is called. This allows multiple sounds to play without cutting each other off. Also, so that the array doesn't infinitely increase in size I've set it to cycle back to index 0 after filling 5 slots in the array. Now everything works perfectly but after 'audioPlayer' has been called a bunch of times, audio suddenly stops and I start getting the 'Error' in the Catch section but my app continues to function normally as if it has just been muted. Can anyone tell me why this is happening?
var audioIndexA = 0
public class AudioPlayer: NSObject {
var playerA = [AVAudioPlayer]()
func playAudio(audioFile audioFile: String){
do{
let path = NSBundle.mainBundle().pathForResource(audioFile, ofType:"wav")
let fileURL = NSURL(fileURLWithPath: path!)
playerA.insert(try AVAudioPlayer(contentsOfURL: fileURL, fileTypeHint: nil), atIndex: audioIndexA)
playerA[audioIndexA].prepareToPlay()
if audioIndexA < 5{
playerA[audioIndexA].play()
audioIndexA++
} else{
playerA[audioIndexA].play()
audioIndexA = 0
}
} catch{
print("Error")
}
}
}
I am trying to create an app in which one of a few objects is randomly placed on the screen wherever you touch. I have my objects a, b, and c and I have them in an array.
(NSArray *) gamePieces {
NSArray *things = [NSArray arrayWithObjects: #"a", #"b", #"c", nil];
return things;
}
And then my touch method.
(void)touchesBegan:(NSArray *)touches withEvent:(UIEvent *)event {
/* Called when a touch begins */
for (UITouch *touch in touches) {
CGPoint location = [touch locationInNode:self];
// code that chooses random object from array
}
}
Normally I would put in an SKSpriteNode and it would make just that one appear on the screen, but I am looking to randomly select one of many. I'm new to programming and I'm not sure if I'm on the right path. If I am, what needs to go in the touch method? If not, what am I doing wrong?
You can get a random item by getting a random index based on the number of elements in the array :
int randomIndex = arc4Random() % things.count;
NSString *thing = things[randomIndex];
// do something with thing
I am new to DirectShow. I, like many others, am trying to create a socket-based P2P streaming solution for a WPF-based card game. I want each player to be able to see each other via small video windows.
My questions are two-fold. The first is How do I lower the frame sample rate and resolution? I believe 320x200 x 15 to 20 fps should be fine. I am using the SampleGrabber callback to grab frame data and send it over the socket; which is actually working with no compression at 640x480 resolution.
My second question is, since each frame contains 921,600 bytes, this really bogs down and I get very slow rendering just across my local WiFi connected LAN. I added a simple MJPEG compression (wanting to switch to h.264 later) and I noticed the bytes drop to around 330-360k. Not a bad improvement.
On the receiving end Do I need to create a custom DirectShow Source Pin in order to serve up the bytes received from the socket so I can attach a decoder and render the bytes in a window?
I just wanted to ask this first since it seems like a lot of work to create a new COM object (haven't done that in about 15 years!), register it, and use/debug it.
Is there perhaps another way?
Also if that is the way to go, should I use a SampleGrabber on the receiving end and create a BitmapSource from the decompressed bytes, or should I allow DirectShow to create a child window? Thing is, I want to have more than one other player and I set an extra byte in the socket to tell what table position they are in. How do I render each position in turn?
For those that are interested, here is how you set the resolution and add an encoder/compressor:
// Create a graph builder
int hr = captureGraphBuilder.SetFiltergraph(graphBuilder);
// Find a capture device (WebCam) and attach it to the graph
sourceFilter = FindCaptureDevice();
hr = graphBuilder.AddFilter(sourceFilter, "Video Capture");
// Get the source output Pin
IPin sourcePin = DsFindPin.ByDirection((IBaseFilter)sourceFilter, PinDirection.Output, 0);
IAMStreamConfig sc = (IAMStreamConfig)sourcePin;
int count;
int size;
sc.GetNumberOfCapabilities(out count, out size);
VideoInfoHeader v;
AMMediaType media2 = null;
IntPtr memPtr = Marshal.AllocCoTaskMem(size);
for (int i = 0; i < count; ++i)
{
sc.GetStreamCaps(i, out media2, memPtr);
v = (VideoInfoHeader)Marshal.PtrToStructure(media2.formatPtr, typeof(VideoInfoHeader));
// Break when width is 160
if (v.BmiHeader.Width == 160)
break;
}
// Set the new media format t0 160 x 120
hr = sc.SetFormat(media2);
Marshal.FreeCoTaskMem(memPtr);
DsUtils.FreeAMMediaType(media2);
// Create a FramGrabber
IBaseFilter grabberF = (IBaseFilter)new SampleGrabber();
ISampleGrabber grabber = (ISampleGrabber)grabberF;
// Set the media type
var media = new AMMediaType
{
majorType = MediaType.Video,
subType = MediaSubType.MJPG
//subType = MediaSubType.RGB24
};
// The media sub type will be MJPG
hr = grabber.SetMediaType(media);
DsUtils.FreeAMMediaType(media);
hr = grabber.SetCallback(this, 1);
hr = graphBuilder.AddFilter(grabberF, "Sample Grabber");
IPin grabberPin = DsFindPin.ByDirection(grabberF, PinDirection.Input, 0);
// Get the MPEG compressor
Guid iid = typeof(IBaseFilter).GUID;
object compressor = null;
foreach (DsDevice device in DsDevice.GetDevicesOfCat(FilterCategory.VideoCompressorCategory))//.MediaEncoderCategory))
{
if (device.Name == "MJPEG Compressor")
{
device.Mon.BindToObject(null, null, ref iid, out compressor);
hr = graphBuilder.AddFilter((IBaseFilter)compressor, "Compressor");
break;
}
string name = device.Name;
}
// This also works!
//IBaseFilter enc = (IBaseFilter)new MJPGEnc();
//graphBuilder.AddFilter(enc, "MJPEG Encoder");
// Get the input and out pins of the compressor
IBaseFilter enc = (IBaseFilter)compressor;
IPin encPinIn = DsFindPin.ByDirection(enc, PinDirection.Input, 0);
IPin encPinOut = DsFindPin.ByDirection(enc, PinDirection.Output, 0);
// Attach the pins: source to input, output to grabber
hr = graphBuilder.Connect(sourcePin, encPinIn);
hr = graphBuilder.Connect(encPinOut, grabberPin);
// Free the pin resources
Marshal.ReleaseComObject(sourcePin);
Marshal.ReleaseComObject(enc);
Marshal.ReleaseComObject(encPinIn);
Marshal.ReleaseComObject(encPinOut);
Marshal.ReleaseComObject(grabberPin);
// Create a render stream
hr = captureGraphBuilder.RenderStream(PinCategory.Preview, MediaType.Video, sourceFilter, null, grabberF);
Marshal.ReleaseComObject(sourceFilter);
Configure(grabber);
I am trying to write a program that will recognize an image on the screen, compare it against a resource library, and then calculate based on the result of the image source.
The first thing that I did was to create the capture screen function which looks like this:
private Bitmap Screenshot()
{
System.Drawing.Bitmap Table = new System.Drawing.Bitmap(88, 40, PixelFormat.Format32bppArgb);
System.Drawing.Graphics g = System.Drawing.Graphics.FromImage(RouletteTable);
g.CopyFromScreen(1047, 44, 0, 0, Screen.PrimaryScreen.Bounds.Size);
return Table;
}
Then, I analyze this picture. The first method I used was to create two for loops and analyze both the bitmaps pixel by pixel. The problem with this method was time, it took a long time to complete 37 times. I looked around and found the convert to bytes and the convert to hash methods. This is the result:
public enum CompareResult
{
ciCompareOk,
ciPixelMismatch,
ciSizeMismatch
};
public CompareResult Compare(Bitmap bmp1, Bitmap bmp2)
{
CompareResult cr = CompareResult.ciCompareOk;
//Test to see if we have the same size of image
if (bmp1.Size != bmp2.Size)
{
cr = CompareResult.ciSizeMismatch;
}
else
{
//Convert each image to a byte array
System.Drawing.ImageConverter ic = new System.Drawing.ImageConverter();
byte[] btImage1 = new byte[1];
btImage1 = (byte[])ic.ConvertTo(bmp1, btImage1.GetType());
byte[] btImage2 = new byte[1];
btImage2 = (byte[])ic.ConvertTo(bmp2, btImage2.GetType());
//Compute a hash for each image
SHA256Managed shaM = new SHA256Managed();
byte[] hash1 = shaM.ComputeHash(btImage1);
byte[] hash2 = shaM.ComputeHash(btImage2);
for (int i = 0; i < hash1.Length && i < hash2.Length&& cr == CompareResult.ciCompareOk; i++)
{
if (hash1[i] != hash2[i])
cr = CompareResult.ciPixelMismatch;
}
}
return cr;
}
After I analyze the two bitmaps in this function, I call it in my main form with the following:
Bitmap Table = Screenshot();
CompareResult success0 = Compare(Properties.Resources.Result0, Table);
if (success0 == CompareResult.ciCompareOk)
{ double result = 0; Num.Text = result.ToString(); goto end; }
The problem I am getting is that once this has all been accomplished, I am always getting a cr value of ciPixelMismatch. I cannot get the images to match, even though the images are identical.
To give you a bit more background on the two bitmaps, they are approximately 88 by 40 pixels, and located at 1047, 44 on the screen. I wrote a part of the program to automatically take a picture of that area so I did not have to worry about the wrong location or size being captured:
Table.Save("table.bmp");
After I took the picture and saved it, I moved it from the bin folder in the project directly to the resource folder and ran the program again. Despite all of this, the result is still ciPixelMismatch. I believe the problem lies within the format that the pictures are being saved as. I believe that despite them being the same image, they are being analyzed in different formats, maybe one of the pictures contains a bit more information than the other which is causing the mismatch. Can somebody please help me solve this problem? I am just beginning with my c# programming, I am 5 days into the learning process, and I am really at a loss for this.
Yours sincerely,
Samuel
I'm trying to add an array of berries to the screen and have them move across to the left. It references a plist which includes the levels. I adapted this from the Ray Winderlich Space Game tutorial.
No sprites are showing up on the screen. :/
-(void)updatePinkBerries:(ccTime)dt
{
CGSize winSize = [CCDirector sharedDirector].winSize;
// if (levelManager.gameState != GameStateNormal)
// return;
//
// if (![levelManager boolForProp:#"SpawnAsteroids"])
// return;
double curTime = CACurrentMediaTime();
if (curTime > nextPinkBerrySpawn)
{
// Figure out the next time to spawn a berry
float spawnSecsLow = [levelManager floatForProp:#"ASpawnSecsLow"];
float spawnSecsHigh = [levelManager floatForProp:#"ASpawnSecsHigh"];
float randSecs = randomValueBetween(spawnSecsLow, spawnSecsHigh);
nextPinkBerrySpawn = randSecs + curTime;
float randY = randomValueBetween(0.0, winSize.height);
float moveDurationLow = [levelManager floatForProp:#"AMoveDurationLow"];
float moveDurationHigh = [levelManager floatForProp:#"AMoveDurationHigh"];
float randDuration = randomValueBetween(moveDurationLow, moveDurationHigh);
// Create a new berry sprite
CCSprite *pinkBerry = [pinkBerries nextSprite];
[pinkBerry stopAllActions];
pinkBerry.visible = YES;
// Set its position to be offscreen to the right
pinkBerry.position = ccp(winSize.width + pinkBerry.contentSize.width/2, randY);
// Move it offscreen to the left, and when it's done, call removeNode
[pinkBerry runAction:
[CCSequence actions:
[CCMoveBy actionWithDuration:randDuration position:ccp(-winSize.width- pinkBerry.contentSize.width, 0)],
[CCCallFuncN actionWithTarget:self selector:#selector(invisNode:)], nil]];
}
}
Check to see if nextPinkBerrySpawn is what it is supposed to be. If curTime is not greater than nextPinkBerrySpawn then it wont spawn anything. The code from the SGSK is very adaptable and you should be able to run it fine if everything is set up right