Is there any way to shorten ReactPlayer progress interval - reactjs

I'm using react-player for a project
<ReactPlayer
ref={p => { this.playerRef = p }}
controls={false}
muted={false}
volume={this.getVolume()}
url="https://vimeo.com/258944633"
progressInterval={30}
playing={this.state.playing}
onStart={this.setStateVidDuration}
onProgress={this.handleProgress}
height="100%"
width={"100%"}
/>
And I'm trying to do a facial recognition overlay based on what frame the video is on. The only way I have to access anything close to the current frame is the onProgress call, and I can set the interval with onProgressInterval. It returns a fraction played which could work for my purposes (not ideal but whatever). Docs here: https://www.npmjs.com/package/react-player
The problem is it currently only runs every 250 ms regardless of what I set onProgressInterval to.
When I read the vimeo player Docs I found that the minimum interval I found matched up with what is suggested in the Docs https://github.com/vimeo/player.js/#timeupdate
Is there any way to get it to tick faster? Even twice as fast would make a big difference.
I am also considering a hacky solution where I updated the real fraction played on each tick and set a smaller interval in between the onProgress ticks where I used the frame rate to estimate the additional frames that have passed since I got the last real reading from the onProgress call.
Edit: It ticks at minimum 250ms when I set it lower than 250 to be clear.

Related

Timer in games for multiple players React

I am making a typeracer on reactjs (I am new in the frontend languages) and so far, so good. The typeracing is working as expected (singleplayer for now). Now what I want to implement is two types of timers:
A typeracing game should last not more than 60 seconds. After 60 seconds I want to end the game.
Before a new game starts, there should be another time for 10 seconds. Which is the same timer after the game ends.
So, in perfect starting scenario, everything would look like this (for now):
I wait 10 seconds (where I have a button to either join the game or not and review the typeracer text), if I've joined the game I have 60 seconds to finish it (if I do, I wait until the end of the timer or until everyone has finished), and then this cycle repeats.
Here comes my question -> how can I set this timer so it is not possible to type in the box:
<div className="box userTypingBox">
<input type="text" onChange={onChange}></input>
</div>
until the timer finished (and also display the seconds left until the game starts/ends). And then how can I set a timer where the box is available for those who has joined and after 60 seconds, the cycle repeats.
Keep in mind (if important) that later I am going to use socket io to make the game for multiple players and multiple players to join one game.
NOT IMPORTANT FOR THE ANSWER: Of course I have also done a research but don't really have something as good as an idea which I could try. The only thing I can think of is make a while cycle for the box and have another class with a timer function which returns either false and true. And if its true the player can type, if its false it should display the 10 second timer.

recharts missing ticks or ticks are displayed wrong

I'm using recharts library to create a chart. The x axis of the chart is the timeline axis and I use timestamps as values for it. What I'm trying to do is to display only 5 ticks(for example) spaced equally. I found a lot of people having similar problems on the library git repository. One fixes was to use scale="time" and setting a minimum tick gap for x axis:
<XAxis dataKey='timestamp'
domain={['dataMin', 'dataMax']}
type = 'number'
scale="time"
minTickGap={120}
tickCount={7} */
tickFormatter = {(unixTime) => moment(unixTime).format('DD MMM')}
/>
I'm using the brush component of the library to zoom in on the chart and the problem with this implementation is that when I zoom in sometimes it displays all the ticks.
Ex. without zoom in:
Ex zooming in:
I found that if I remove the scale="time" parameter and set a specific tickCount and set interval to 0 the problem that occurs when I zoom in disappears but 1 tick is missing or it appears only when I zoom in.
<XAxis dataKey='timestamp'
domain={['dataMin', 'dataMax']}
type = 'number'
interval={0}
tickCount={7}
tickFormatter = {(unixTime) => moment(unixTime).format('DD MMM')}
/>
Does anyone now how can I fix the issues mentioned above? I don't think I can use predefined ticks based on maximum and minimum values of the axis because those values becomes irrelevant when I zoom in.
There is no cure for it as far as I know. You could mention all your tick explicitly in ticks param like ticks={[10,20,50,200,500,1000]} and they will do just fine
Another workaround for me is to set domain as
domain= {['auto','auto']}
and it shows all the ticks correctly.

Using requestAnimationFrame in React

I am new to react native and I trying to optimize performance.
My Touch events are very slow and I was going through RN documentation on performance and they have mentioned to use requestAnimationFrame with this example
handleOnPress() {
// Always use TimerMixin with requestAnimationFrame, setTimeout and
// setInterval
this.requestAnimationFrame(() => {
this.doExpensiveAction();
});
}
Now, this description sounds very vague and hard for me to comprehend its usage
Like, I have this touchable event in my RN app
<TouchableWithoutFeedback onPress={() => this.touched(this.props.coinShortName)}>
Which calls this method
touched = (id) => {
this.setState({selectedPostId: id})
if (this.props.coinShortName == this.state.selectedPostId ) {
this.setState({stateToDisplay: !this.state.stateToDisplay})
}
}
Question: Can someone please tell me on how I need to/should use it in my app?
I'm going to convert my comment into an answer, so I can format it better and hopefully help someone in the future too. I don't expressly recommend marking my answer as correct, but I think this answer should be here alongside this question.
This article here should give you some backstory on requestAnimationFrame: http://www.javascriptkit.com/javatutors/requestanimationframe.shtml.
I would recommend reading the article I linked above and then read my answer after.
I will just explicitly mention that requestAnimationFrame could appear similar to setTimeout(() => {}, 0), but if you had a Zack Morris phone made in 1985, its "as soon as possible" might be 5 seconds later, thus making your animation look terrible, similar to when your character lags across the screen in a video game. The function may have been called at the correct time, but it did not actually execute at the correct time.
It can be helpful to imagine a collection phase and a rendering phase. I'm sorry, I don't know the exact terms for this stuff, but human eyes see smooth movement at I think 20 FPS, but what that means is you have 20 "frames", so it's like calculating something 20 times. It's like collecting a bunch of kids and pushing them into a bus 20 times per second. Pushing them into the bus is an event, and it's analogous to repainting your screen. Sometimes kids can get left behind and extra kids picked up next time, so you can imagine the gains to perceived smoothness of flow over time.
It's important to note that optimizations are being made with respect to the next repaint, or the next time the screen 'changes'. requestAnimationFrame does work under the hood to ensure the animation occurs at the correct time and is smooth, meaning the pixels were where they were supposed to be at the right time. (I think you would derive a lot of meaning if you checked out the definitions for "what is a janky animation", and look at some of the discussion around that. I mention that because we want to understand more about the repainting process and what kinds of things are important and why)
I recall that requestAnimationFrame can ditch calculations that would occur too late. For example, if you click the button and a pixel goes from 0% to 25% to 50% to 75% to 100% (some arbitrary distance calculation). We could say that after 1 second, the pixel should have travelled 50% of the distance and after 2 seconds, it should be at 100%, the final resting place.
It's more important that the pixels are in the correct place at the correct time than it is for them to travel to exactly every place they were supposed to. requestAnimationFrame is helping you do this. If the screen is about to repaint, and "it" needs to run a calculation that would take too long, "it" just ignores it and skips to the next frame. It's like trimming fat to keep on pace and therefore, avoid jank.
requestAnimationFrame is a solution for the same challenges whether it's in your web browser or iOS or Android. They all do this process of painting the screen over and over again. You could start calculating something needed for the next repaint but start too late so it's not done when the next repaint occurs.
Imagine your animation was smooth but your phone received 20 push notifications all of a sudden that bogged down the CPU, causing your animation to be delayed by 16.7 milliseconds. Rather than display the pixel at the correct place at the wrong time, requestAnimationFrame helps by making the pixel be in the correct place at the correct time, but it may do some magic and not even try to paint the pixel sometimes when it would have otherwise, thus saving performance and increasing perceived smoothness.
I just realized this is a wall of text, but I think it will be informational.
These repaints occur about 60 frames per second, so requestAnimationFrame could fire like 60 times a second when it calculates is the most optimal time. There are 1000 milliseconds in 1 second, so 60 FPS is one frame every 16.7ms. If the human eye perceives smoothness at 20FPS, then it means you could in theory repaint every 45ms or 30% as much, and the animation would still be smooth.
My definitions may be inaccurate, but I hope they can help give you a sense what is happening.
As mentioned , you need to wrap your expensive action in an instance of requestAnimationFrame.
Usage
<TouchableWithoutFeedback onPress={() => this.touched(this.props.coinShortName)}>
touched = (id) => {
requestAnimationFrame(() => {
this.setState({selectedPostId: id})
if (this.props.coinShortName == this.state.selectedPostId ) {
this.setState({stateToDisplay: !this.state.stateToDisplay})
}
});
}
<TouchableWithoutFeedback onPress={this.handlePress}>
handlePress = () => {
this.requestAnimationFrame(() => {
this.touched(this.props.coinShortName)
});
}
it would be event better if you removed the useless id parameter in the touched function by using directly this.props.coinShortName inside so you could write
handlePress = () => {
this.requestAnimationFrame(this.touched);
}
Anyway, the touched function doesn't seem to be really expensive so I don't know if it will solve your performance issue

How do I reliably pause the state of a game?

So I have a couple instances where I want to be able to 'freeze' the state of my game. It's a top-down scroller, and I want to give the player the ability to pause the scrolling of the screen for a short time by using a powerup (if you fall to the bottom of the screen you die). I also want to pause the game as it is starting, and draw a 3, 2, 1, go! to give the player time to get ready, because right now as soon as you hit play, the screen starts scrolling.
I have been using Timer to accomplish this, however it doesn't work if I want to freeze the screen on consecutive occasions. Like if a player uses a freeze, the screen sucesssfully freezes, but if they quickly use another freeze, it doesn't work. There seems to be an unintended cool-down. I have a similar problem for the 'intro delay' I explained earlier. For some reason it only works on the first 2 levels. Here is how I am using Timer.
if(gameState != STATE.frozen) {
camera.translate(0, (float) scrollSpeed);
staminaBar.setPosition(staminaBar.getX(), (float) (staminaBar.getY()+scrollSpeed));
staminaMeter.setPosition(staminaMeter.getX(), (float) (staminaMeter.getY()+scrollSpeed));
healthBar.setPosition(healthBar.getX(), (float) (healthBar.getY()+scrollSpeed));
healthMeter.setPosition(healthBar.getX(), (float) (healthMeter.getY()+scrollSpeed));
boostBar.setPosition(boostBar.getX(), (float) (boostBar.getY()+scrollSpeed));
boostMeter.setPosition(boostMeter.getX(), (float) (boostMeter.getY()+scrollSpeed));
screenCeiling += (float) scrollSpeed;
screenFloor += (float) scrollSpeed;
}
else {
Timer.schedule(new Task() { //freeze the screen for 5 seconds
#Override
public void run() {
gameState = STATE.playing;
}
}, 5);
}
From what I understand, it waits 5 second before resuming the game to the 'playing' state. But like I said, this only works when activated between large intervals and I don't know why. Is there a better way I can be doing this?
As for the intro delay, this may be a question better asked seperate, but I use the same method, but it doesn't let me draw sprites over my tiledmap, so if anyone knows how to do that please include it in your response
Assuming the code you posted is in your render loop, then whenever you are not in the frozen state, you are creating a new timer task on every frame. So if you freeze for 5 seconds and your game is running at 60fps, you will create 300 timer tasks, each of which is going to force the game to go back to playing state. The last one won't fire until 5 seconds after the first one fires, so there will be a five second "cooldown" during which you cannot change the state to anything besides playing, because there will be another timer task firing on every frame during that time.
You need to ensure that you only create one timer task, only when you first enter frozen state.
I do have a suggestion...instead of using a state to freeze the game, use a variable that's multiplied by scrollSpeed. Change that variable from one to zero when the player uses the powerup. Then you can do fancy stuff like quickly interpolating from one to zero so the speed change isn't so abrupt. And it will probably make your code simpler since there would be one less state that must be handled differently in the algorithm.
Check your gameState variable in the render method and if the game is playing, then update the game as usual and draw it.
If the game is not playing then skip the game's update method and create a time delay from the current time:
endTime = TimeUtils.millis()+5000;
Then each time through the render method check to see if current time is greater than the end time. When the current time is past your delay time, set gameState back to playing and have the game go back to updating.
You'll have to have another boolean flag so you only set the endTime once (you don't want to keep resetting this each time through the render loop), or if "STATE" is an enum, then include an option for "justPaused" for the exact frame that you pause the game, set the end time, then set STATE to "notPlaying".
You can also use this to create an alternative "update" method where you can update your countdown sprites, but not update the game itself. When the game is playing this other update method will be skipped.

Given an audio stream, find when a door slams (sound pressure level calculation?)

Not unlike a clap detector ("Clap on! clap clap Clap off! clap clap Clap on, clap off, the Clapper! clap clap ") I need to detect when a door closes. This is in a vehicle, which is easier than a room or household door:
Listen: http://ubasics.com/so/van_driver_door_closing.wav
Look:
It's sampling at 16bits 4khz, and I'd like to avoid lots of processing or storage of samples.
When you look at it in audacity or another waveform tool it's quite distinctive, and almost always clips due to the increase in sound pressure in the vehicle - even when the windows and other doors are open:
Listen: http://ubasics.com/so/van_driverdoorclosing_slidingdoorsopen_windowsopen_engineon.wav
Look:
I expect there's a relatively simple algorithm that would take readings at 4kHz, 8 bits, and keep track of the 'steady state'. When the algorithm detects a significant increase in the sound level it would mark the spot.
What are your thoughts?
How would you detect this event?
Are there code examples of sound pressure level calculations that might help?
Can I get away with less frequent sampling (1kHz or even slower?)
Update: Playing with Octave (open source numerical analysis - similar to Matlab) and seeing if the root mean square will give me what I need (which results in something very similar to the SPL)
Update2: Computing the RMS finds the door close easily in the simple case:
Now I just need to look at the difficult cases (radio on, heat/air on high, etc). The CFAR looks really interesting - I know I'm going to have to use an adaptive algorithm, and CFAR certainly fits the bill.
-Adam
Looking at the screenshots of the source audio files, one simple way to detect a change in sound level would be to do a numerical integration of the samples to find out the "energy" of the wave at a specific time.
A rough algorithm would be:
Divide the samples up into sections
Calculate the energy of each section
Take the ratio of the energies between the previous window and the current window
If the ratio exceeds some threshold, determine that there was a sudden loud noise.
Pseudocode
samples = load_audio_samples() // Array containing audio samples
WINDOW_SIZE = 1000 // Sample window of 1000 samples (example)
for (i = 0; i < samples.length; i += WINDOW_SIZE):
// Perform a numerical integration of the current window using simple
// addition of current sample to a sum.
for (j = 0; j < WINDOW_SIZE; j++):
energy += samples[i+j]
// Take ratio of energies of last window and current window, and see
// if there is a big difference in the energies. If so, there is a
// sudden loud noise.
if (energy / last_energy > THRESHOLD):
sudden_sound_detected()
last_energy = energy
energy = 0;
I should add a disclaimer that I haven't tried this.
This way should be possible to be performed without having the samples all recorded first. As long as there is buffer of some length (WINDOW_SIZE in the example), a numerical integration can be performed to calculate the energy of the section of sound. This does mean however, that there will be a delay in the processing, dependent on the length of the WINDOW_SIZE. Determining a good length for a section of sound is another concern.
How to Split into Sections
In the first audio file, it appears that the duration of the sound of the door closing is 0.25 seconds, so the window used for numerical integration should probably be at most half of that, or even more like a tenth, so the difference between the silence and sudden sound can be noticed, even if the window is overlapping between the silent section and the noise section.
For example, if the integration window was 0.5 seconds, and the first window was covering the 0.25 seconds of silence and 0.25 seconds of door closing, and the second window was covering 0.25 seconds of door closing and 0.25 seconds of silence, it may appear that the two sections of sound has the same level of noise, therefore, not triggering the sound detection. I imagine having a short window would alleviate this problem somewhat.
However, having a window that is too short will mean that the rise in the sound may not fully fit into one window, and it may apppear that there is little difference in energy between the adjacent sections, which can cause the sound to be missed.
I believe the WINDOW_SIZE and THRESHOLD are both going to have to be determined empirically for the sound which is going to be detected.
For the sake of determining how many samples that this algorithm will need to keep in memory, let's say, the WINDOW_SIZE is 1/10 of the sound of the door closing, which is about 0.025 second. At a sampling rate of 4 kHz, that is 100 samples. That seems to be not too much of a memory requirement. Using 16-bit samples that's 200 bytes.
Advantages / Disadvantages
The advantage of this method is that processing can be performed with simple integer arithmetic if the source audio is fed in as integers. The catch is, as mentioned already, that real-time processing will have a delay, depending on the size of the section that is integrated.
There are a couple of problems that I can think of to this approach:
If the background noise is too loud, the difference in energy between the background noise and the door closing will not be easily distinguished, and it may not be able to detect the door closing.
Any abrupt noise, such as a clap, could be regarded as the door is closing.
Perhaps, combining the suggestions in the other answers, such as trying to analyze the frequency signature of the door closing using Fourier analysis, which would require more processing but would make it less prone to error.
It's probably going to take some experimentation before finding a way to solve this problem.
You should tap in to the door close switches in the car.
Trying to do this with sound analysis is overengineering.
There are a lot of suggestions about different signal processing
approaches to take, but really, by the time you learn about detection
theory, build an embedded signal processing board, learn the processing
architecture for the chip you chose, attempt an algorithm, debug it, and then
tune it for the car you want to use it on (and then re-tune and re-debug
it for every other car), you will be wishing you just stickey taped a reed
switch inside the car and hotglued a magnet to the door.
Not that it's not an interesting problem to solve for the dsp experts,
but from the way you're asking this question, it's clear that sound
processing isn't the route you want to take. It will just be such a nightmare
to make it work right.
Also, the clapper is just an high pass filter fed into a threshold detector. (plus a timer to make sure 2 claps quickly enough together)
There is a lot of relevant literature on this problem in the radar world (it's called detection theory).
You might have a look at "cell averaging CFAR" (constant false alarm rate) detection. Wikipedia has a little bit here. Your idea is very similar to this, and it should work! :)
Good luck!
I would start by looking at the spectral. I did this on the two audio files you gave, and there does seem to be some similarity you could use. For example the main difference between the two seems to be around 40-50Hz. My .02.
UPDATE
I had another idea after posting this. If you can, add an accelerometer onto the device. Then correlate the vibrational and acoustic signals. This should help with cross vehicle door detection. I'm thinking it should be well correlated since the sound is vibrationally driven, wheres the stereo for example, is not. I've had a device that was able to detect my engine rpm with a windshield mount (suction cup), so the sensitivity might be there. (I make no promises this works!)
(source: charlesrcook.com)
%% Test Script (Matlab)
clear
hold all %keep plots open
dt=.001
%% Van driver door
data = wavread('van_driver_door_closing.wav');
%Frequency analysis
NFFT = 2^nextpow2(length(data));
Y = fft(data(:,2), NFFT)/length(data);
freq = (1/dt)/2*linspace(0,1,NFFT/2);
spectral = [freq' 2*abs(Y(1:NFFT/2))];
plot(spectral(:,1),spectral(:,2))
%% Repeat for van sliding door
data = wavread('van_driverdoorclosing.wav');
%Frequency analysis
NFFT = 2^nextpow2(length(data));
Y = fft(data(:,2), NFFT)/length(data);
freq = (1/dt)/2*linspace(0,1,NFFT/2);
spectral = [freq' 2*abs(Y(1:NFFT/2))];
plot(spectral(:,1),spectral(:,2))
The process for finding distinct spike in audio signals is called transient detection. Applications like Sony's Acid and Ableton Live use transient detection to find the beats in music for doing beat matching.
The distinct spike you see in the waveform above is called a transient, and there are several good algorithms for detecting it. The paper Transient detection and classification in energy matters describes 3 methods for doing this.
I would imagine that the frequency and amplitude would also vary significantly from vehicle to vehicle. Best way to determine that would be taking a sample in a Civic versus a big SUV. Perhaps you could have the user close the door in a "learning" mode to get the amplitude and frequency signature. Then you could use that to compare when in usage mode.
You could also consider using Fourier analysis to eliminate background noises that aren't associated with the door close.
Maybe you should try to detect significant instant rise in air pressure that should mark a door close. You can pair it with this waveform and sound level analysis and these all might give you a better result.
On the issue of less frequent sampling, the highest sound frequency which can be captured is half of the sampling rate. Thus, if the car door sound was strongest at 1000Hz (for example) then a sampling rate below 2000Hz would lose that sound entirely
A very simple noise gate would probably do just fine in your situation. Simply wait for the first sample whose amplitude is above a specified threshold value (to avoid triggering with background noise). You would only need to get more complicated than this if you need to distinguish between different types of noise (e.g. a door closing versus a hand clap).

Resources