put live audio input data into array - arrays

Hey I am using GetUserMedia() to capture audio input from user's microphone. Meanwhile I want to put captured values into an array so I can manipulate with them. I am using the following code but the problem is that my array gets filled with value 128 all the time (I print the results in console for now), and I can't find my mistake. Can someone help me find my mistake?
//create a new context for audio input
context = new webkitAudioContext();
var analyser = null;
var dataarray = [];
getLiveInput = function() {
navigator.webkitGetUserMedia({audio: true},onStream,onStreamError);
};
function onStream(stream)
{
var input = context.createMediaStreamSource(stream);
analyser = context.createAnalyser();
var str = new Uint8Array(analyser.frequencyBinCount);
analyser.getByteTimeDomainData(str);
for (var i = 0; i < str.length; i++) {
var value = str[i];
dataarray.push(value);
console.log(dataarray)
}//end for loop
}//end function
function onStreamError(e) {
console.error('Streaming failed: ', e);
};

The values returned from getByteTimeDomainData are 8 bit integers, from 0 to 255. 128, which is half way, basically means "no signal". It is the equivalent of 0 in PCM audio data from -1 to 1.
But ANYWAY - there are a couple problems:
First, you're never connecting the input to the analyser. You need input.connect(analyser) before you call analyser.getByteTimeDomainData().
The second problem isn't with your code so much as it's just an implementation issue.
Basically, the gotStream function only gets called once - and getByteTimeDomainData only returns data for 1024 samples worth of audio (a tiny fraction of a second). The problem is, this all happens so quickly and for such a short period of time after the stream gets created, that there's no real input yet. Try wrapping the analyser.getByteTimeDomainData() call and the loop that follows it in a 1000ms setTimeout and then whistle into your microphone as soon as you give the browser permission to record. You should see some values other than 128.
Here's an example: http://jsbin.com/avasav/5/edit

Related

NAudio trimming mp3 file

I'm trying to trim a mp3 file.
using this code:
private void TrimMp3(string open, string save)
{
using (var mp3FileReader = new Mp3FileReader(open))
using (var writer = File.Create(save))
{
var startPostion = TimeSpan.FromSeconds(60);
var endPostion = TimeSpan.FromSeconds(90);
mp3FileReader.CurrentTime = startPostion;
while (mp3FileReader.CurrentTime < endPostion)
{
var frame = mp3FileReader.ReadNextFrame();
if (frame == null) break;
writer.Write(frame.RawData, 0, frame.RawData.Length);
}
}
}
"open" is the file I'm trimming and "save" is the location I'm saving.
The trimming works but not fully. The new file does start from 60 seconds but it keeps going and not stopping at 90 seconds. For example if the file is 3 minutes it will start at 1 minute and end at 3. Its like the while is always true. What am I doing wrong here?
Thanks in advance!
I have no idea what your Mp3FileReader is doing there. But your while loop looks odd. Does mp3FileRead.ReadNextFrame() also change mp3FileReader.CurrentTime ? If not then there is your problem.
You should atleast do mp3FileReader.CurrentTime + 1Frame. Otherwise your currenttime is never changed and loop will always be true
In NAudio 1.8.0, Mp3FileReader.ReadNextFrame does not progress CurrentTime, although I checked in a fix for that recently.
So you can either get the latest NAudio code, or make use of the SampleCount property on each Mp3Frame to accurately keep track of how far through you are yourself.

Append function overwrites existing data in slice

I wrote a small application which records data from a sound card and stores the data in an array for later processing.
Whenever new data is available, portaudio executes the callback record. Within the callback I append the data to the array RecData.data.
The golang builtin function append adds as expected another element to the slice, but for whatever reason also overwrites all existing elements within the array with exactly the same data.
I have been trying to isolate the problem for more than two days, without success.
Here is a stripped down version of the code, which works and shows the problem:
package main
import (
"fmt"
"time"
// "reflect"
"github.com/gordonklaus/portaudio"
)
type RecData struct{
data [][][]float32
}
func main() {
var inputChs int = 1
var outputChs int = 0
var samplingRate float64 = 48000
var framesPerBuffer int = 3 //for test purpose that low. Would normally be 1024 or 2048
rec := RecData{make([][][]float32, 0, 1000)}
portaudio.Initialize()
stream, err := portaudio.OpenDefaultStream(inputChs, outputChs, samplingRate, framesPerBuffer, rec.record)
if err != nil {
fmt.Println(err)
}
defer stream.Close()
stream.Start()
for {
time.Sleep(time.Millisecond * 10)
}
}
// callback which gets called when new data is in the buffer
func (re *RecData)record(in [][]float32) {
fmt.Println("Received sound sample: ")
fmt.Println(in)
re.data = append(re.data, in)
fmt.Println("Content of RecData.data after adding received sound sample:")
fmt.Println(re.data, "\n")
time.Sleep(time.Millisecond * 500) //limit temporarily the amount of data read
// iterate over all recorded data and compare them
/*
for i, d := range re.data{
if reflect.DeepEqual(d, in){
fmt.Printf("Data at index %d is the same as the recorded one, but should not be!\n", i )
}
}*/
}
2. Update
This is the application output:
Received sound sample:
[[0.71575254 1.0734825 0.7444282]]
Content of RecData.data after adding received sound sample:
[[[0.71575254 1.0734825 0.7444282]]]
Received sound sample:
[[0.7555193 0.768355 0.6575008]]
Content of RecData.data after adding received sound sample:
[[[0.7555193 0.768355 0.6575008]] [[0.7555193 0.768355 0.6575008]]]
Received sound sample:
[[0.7247052 0.68471473 0.6843796]]
Content of RecData.data after adding received sound sample:
[[[0.7247052 0.68471473 0.6843796]] [[0.7247052 0.68471473 0.6843796]] [[0.7247052 0.68471473 0.6843796]]]
Received sound sample:
[[0.6996536 0.66283375 0.67252487]]
Content of RecData.data after adding received sound sample:
[[[0.6996536 0.66283375 0.67252487]] [[0.6996536 0.66283375 0.67252487]] [[0.6996536 0.66283375 0.67252487]] [[0.6996536 0.66283375 0.67252487]]]
.... etc ....
As we one can see, over time, the size of the slice is growing, but instead of just appending the data, the data in the array gets also overwritten.
This should not happen. portaudio provides in the callback a [][]float32 with the audio sample recorded from the sound card. As you can see they are always different.
As mentioned, the code above is a stripped down version of my application. Usually I would record lets say 5 seconds, and then perform a Fast Fourier Transformation (FFT) over the samples to calculate the spectrum. I left this part away since it has no impact on this particular problem.
I would very much appreciate any help. Maybe somebody can point me out what I'm doing wrong.
Thanks!
The buffer passed into the callback is reused by the portaudio package, so you are appending the same slice structure to your data slice. Each time the buffer allocated by portaudio overwrites the data, you see the results in every element of your data slice.
You will need to allocate new slices and make a copy of the data:
func (re *RecData) record(in [][]float32) {
buf := make([][]float32, len(in))
for i, v := range in {
buf[i] = append([]float32(nil), v...)
}
re.data = append(re.data, buf)
Example:
https://play.golang.org/p/cF57lQIZFU

iOS9 Beta and MusicTrackLoopInfo

Has anyone been able to loop a MIDI file without problems on IOS9 Beta?
As soon as I try to loop by setting numberOfLoops to 0 in MusicTrackLoopInfo, it locks up the app by sending random MIDI to the player. I've reported it, but am wondering if anyone has found a work around. The same code works perfectly under all other iOS versions.
MusicTrackLoopInfo loopInfo;
loopInfo.loopDuration = loopLength;
loopInfo.numberOfLoops = 0;
OK I just heard iOS9 will ship with this bug in it. Terrible.
Here is a work around.
Don't set numberOfLoops at all, OR set numberOfLoops = 1; // means loop once
Now make a variable (i.e. myVariableToKeepTrackOfAddedCopies) that keeps track of the number of times you will actually perform the following:
In your MIDIReadProc at some point BEFORE the track has finished playing, do the following:
// Copy the track to itself - effectively doubling the length
MusicTrack theTrack=nil;
MusicTrackGetProperty(theTrack, kSequenceTrackProperty_TrackLength, &trackLen, &trackLenLen);
trackLen = 4.0; //<-- this is your real track length
MusicTrackCopyInsert(theTrack, 0, trackLen, theTrack, 0);
myVariableToKeepTrackOfAddedCopies++;
So now your track is twice as long before it ends and the track will continue. This will work the same as looping except you are taking up more memory since you are making the track length longer after each iteration.
When you stop the sequence/track, cut the track back to the original size.
MusicTrackCut(theTrack, 4.0, 4.0 + (4.0*myVariableToKeepTrackOfAddedCopies));
MusicTrackGetProperty(theTrack, kSequenceTrackProperty_TrackLength, &trackLen, &trackLenLen);
Irritating, but it works. I just verified on iOS9 beta 5. Hope it helps.
This is fixed as of iOS release 9.2
Oddly enough, the tempo track does not seem to have this problem. The following code does not lock up for me:
MusicTrack tempoTrack;
OSSTATUS = MusicSequenceGetTempoTrack(self.sequence, &tempoTrack);
SafeMusicTrackClear(tempoTrack); //calls into MusicTrackClear
MusicTrackNewExtendedTempoEvent(tempoTrack, 0, self.tempo * self.tempoMultiplier);
MIDIMetaEvent timeSignatureMetaEvent;
timeSignatureMetaEvent.metaEventType = 0x58;
timeSignatureMetaEvent.dataLength = 4;
timeSignatureMetaEvent.data[0] = 1;
timeSignatureMetaEvent.data[1] = 4;
timeSignatureMetaEvent.data[2] = 0x18;
timeSignatureMetaEvent.data[3] = 0x08;
MusicTrackNewMetaEvent(tempoTrack, 0, &timeSignatureMetaEvent);
MusicTrackLoopInfo loopInfo;
loopInfo.loopDuration = 0.25f;
loopInfo.numberOfLoops = 0;
MusicTrackSetProperty(tempoTrack, kSequenceTrackProperty_LoopInfo, &loopInfo, sizeof(loopInfo));
Unfortunately, it does not seem that the tempo track can actually play notes.
UPDATE:
After a few hours of digging around and trying to figure out a better solution to the problem, I settled on manually looping by sending a user event at the end of my sequence.
My sequence is created in a method...
-(void) loadPacketsForLoopingSequence {
SafeMusicTrackClear(loopingTrack); //calls into MusicTrackClear
// calculate timestampToPlaySequenceAt -- the starting point of the current sequence iteration, probably in the past, based on MusicPlayerGetTime and the length of the sequence -- here
// calculate timestampToPlayNextSequenceAt -- the starting point of the next sequence iteration, based on MusicPlayerGetTime and the length of the sequence -- here
// a single iteration of the notes get added to loopingTrack here, starting at timestampToPlaySequenceAt
MusicEventUserData event;
event.length = 1;
event.data[0] = 0xab; //arbitrary designation
// -0.5 to make sure we still have time to do the next step in the callback
MusicTrackNewUserEvent(loopingTrack, timestampToPlayNextSequenceAt - 0.5, &event);
}
...which is called again in the callback:
void sequenceCallback(void* inClientData,
MusicSequence inSequence,
MusicTrack inTrack,
MusicTimeStamp inEventTime,
const MusicEventUserData* inEventData,
MusicTimeStamp inStartSliceBeat,
MusicTimeStamp inEndSliceBeat) {
CSMidiMusicPlayer* musicPlayer = (CSMidiMusicPlayer*)inClientData;
[musicPlayer loadPacketsForLoopingSequence];
}
The callback has to be registered during sequence init using MusicSequenceSetUserCallback.
The -0.5 kludge could probably be eliminated altogether by examining the parameters in sequenceCallback and modifying loadPacketsForLoopingSequence to accept a parameter, but I haven't gotten that far yet.
I like this solution because it stays in MIDI time and doesn't modify the MIDI file in unexpected, stateful ways. (New notes basically get streamed in when you get close enough to a loop marker.)

as3 array of objects - movement with constant speed

Ok, so I have some experience with as3 and some of the basics. But this problem has been stumping me for so long. I tried to do a workaround based on what I currently know about as3. But somehow either i get an error message or it just doesn't do anything at all. Here is the code that i'm trying to solve.
var zombieCount:Array = new Array();
var helltime:Timer = new Timer(1500);
helltime.addEventListener(TimerEvent.TIMER, spawnzombies)
helltime.start();
function spawnzombies(happened:TimerEvent){
var zombie1:Zombie = new Zombie();
zombieCount.push(zombie1);
stage.addChild(zombieCount[zombieCount.length - 1]);
zombie1.x = 135 + (330*Math.random())
zombie1.y = -29
stage.addEventListener(Event.ENTER_FRAME, move_zombie)
function move_zombie(happened:Event){
for(var i:int; i < zombieCount.length; i++){
zombieCount[i].y = zombieCount[i].y + 1;
if(zombieCount[i].hitTestObject(border)){
stage.removeChild(zombieCount[i]);
zombieCount.shift();
trace(zombieCount.length);
}
}
}
}
While this may not be inclusive of everything wrong, here are at least a few of the issues I see.
Inline function issue:
Inside your timer tick handler (spawnZombies), you create an inline function called move_zombie and then add an enter frame handler that calls that function.
The issue here, is that every tick of the timer, will then create a whole new copy of that function, and add ANOTHER enter frame handler. This will create huge problems after a few timer ticks.
Break that move_zombie function OUT OF the spawn function:
eg:
helltime.addEventListener(TimerEvent.TIMER, spawnzombies)
helltime.start();
stage.addEventListener(Event.ENTER_FRAME, move_zombie);
function move_zombie(......
function spawnzombies(.....
Iteration issue:
In your for loop:
for(var i:int; i < zombieCount.length; i++){
zombieCount[i].y = zombieCount[i].y + 1;
if(zombieCount[i].hitTestObject(border)){
stage.removeChild(zombieCount[i]);
zombieCount.shift();
trace(zombieCount.length);
}
}
You are not initializing your i value. While this will default it to 0, it's still a good idea for readability to initialize it.
So your iterating forward from 0 to the end of the array. However, if your hit test succeeds, you then use the shift method of the array. This removes the first item of the array (irrespective of what value i is at the time). This will remove the wrong item, plus mess up what zombieCount[i] refers to (because the amount of items has now changed after doing shift, so the next iteration zombieCount[i] will be a reference to same item as the previous iteration).
Instead of what you're currently doing, use the splice method to remove, and iterate backwards so your index doesn't get out of whack.
for(var i:int=zombieCount.length-1;i >=0;i--){
zombieCount[i].y += 1; //move it down 1 pixel
if(zombieCount[i].hitTestObject(border)){
stage.removeChild(zombieCount[i]);
zombieCount.splice(i,1); //remove the item at the current index (do this instead of shift)
trace(zombieCount.length);
}
}

Clearing my array leaves blank array positions

So I'm facing a problem with an AS3 class that's not operating the way I need it to. It's a simple problem, but a complex set of methods that cause it.
Firstly, the 'quiz' I'm building has 6 questions loaded as external SWFs inside of a Shell, run by a class. First we declared a var "a_quiz" to hold 6 values pushed from the external SWFs. These 6 values are reduced to a string and then checked against another array that contains the correct answers. The following loadQuiz function is designed to launch one of three random quizes and clear the a_quiz array so it can take new answers:
public function loadQuiz():void {
a_quiz.length=0;
trace("loadQuiz");
n_quizCorrect = n_quizScore = 0;
n_currentQuiz = Math.floor(Math.random() * (n_totalTopics - n_quizStart + 1) + n_quizStart);
loadTopic(n_currentQuiz);
}
Now I've tested the Shell and confirmed that the trace "loadQuiz" fires every time. And the first time you load the quiz, everything behaves as it should. The 6 questions trace correct or incorrect responses and push 6 binary values into the a_quiz array. The output looks like this:
loadQuiz
incorrect
correct
incorrect
incorrect
incorrect
incorrect
0,1,0,0,0,0
you failed
Then I jump back to the main menu and launch again. This deploys the loadQuiz function all over again. The first line of the function:
a_quiz.length=0;
should be emptying the a_quiz array to accept new answers to mark against. But when I complete the quiz I get this:
loadQuiz
correct
correct
correct
correct
correct
correct
,,,,,,1,1,1,1,1,1
you failed
For some reason beyond my understanding, the push values are stacking on top of empty positions, so when the strings compare...they won't match. What is going on here?
The push function:
function handleClick(evt:MouseEvent):void{
var tempCORRECT = a_answerSheet.toString();
var tempSELECTION = a_selected.toString();
//
if(tempSELECTION == tempCORRECT){
trace('correct');
parentObj1.a_quiz[ parentObj1.n_currentQuestion - 1 ] = 1;
}else{
trace('incorrect');
parentObj1.a_quiz[ parentObj1.n_currentQuestion - 1 ] = 0;
}
parentObj1.n_currentQuestion ++;
// GOTO NEXT SLIDE
parentObj1.LOADNEXT('up');
}
The problem originated in the loading of the Quiz itself.
as you can see on the handleClick function:
parentObj.n_currentQuestion++
was increasing an integer variable on the parentObj's timeline called n_currentQuestion. The problem then, originated in how the quiz was logging n_currentQuestion when the quiz loads with this function:
public function loadQuiz():void {
a_quiz= [];
trace("loadQuiz");
n_quizCorrect = n_quizScore = 0;
n_currentQuiz = Math.floor(Math.random() * (n_totalTopics - n_quizStart + 1) + n_quizStart);
loadTopic(n_currentQuiz);
}
So to correct the error, I needed to add a line to the loadQuiz, so that it would always load with n_currentQuestion set to 1.
public function loadQuiz():void {
a_quiz= [];
n_currentQuestion = 1;
trace("loadQuiz");
n_quizCorrect = n_quizScore = 0;
n_currentQuiz = Math.floor(Math.random() * (n_totalTopics - n_quizStart + 1) + n_quizStart);
loadTopic(n_currentQuiz);
}

Resources