I have an array of observables. Each of them will make a http call to a REST endpoint and return a result so I can update the UI.
I am using zip to run them all like this:
Observable.zip(allTransactions).subscribe(result=> {blab});
In subscribe, I update a page-level collection, so UI gets updated via 2-way binding (angular).
however, there are a few problems:
1) when I construct each observable in the array, I added .delay(1000) to it, so I expect each run will delay at least 1 second to the previous one. In fact, that's not true. Based on my log, it seems all those transactions were fired at the same time. But the subscribe was delayed on second. I really need them run in sequence, as the order I setup the array, because I have some dependency in those transactions. Running all together won't work for me.
2) zip doesn't seem to guarantee to bring back my results in ordered. So my UI is totally in random ordered. Because I was doing this.items.push(result), where items is a variable being bound to the UI.
I am currently trying to merge all the transactions and add an empty observable with delay between every 2 transactions (still working on it).
Can anyone provide any suggestion what other alternatives I can do? or a better way I can try?
Thanks
1) You are correct that adding .delay(1000) to all observables will not make them wait for the previous one. The delay operator will delay the execution from the moment you subscribe to them, and since you subscribe to them all at the same time and delay them for the same amount of time, they will all execute at the same time.
If you want to execute the observable in sequence, and wait for one to finish before proceeding to the next then use the flatMap operator:
obs1.get()
// First call
.flatMap(response1 => {
return obs2.get(response1.something);
})
.subscribe(response2 => {
// Result from second call
});
2) Looking at the zip documentation the result should return in an ordered list:
Merges the specified observable sequences or Promises into one
observable sequence by using the selector function whenever all of the
observable sequences have produced an element at a corresponding
index. If the result selector function is omitted, a list with the
elements of the observable sequences at corresponding indexes will be
yielded.
But as you have noted: the calls are all executed simultaneously and one observable does not wait for the next one before starting.
So the call:
Observable.zip(obs1, obs2, obs3).subscribe(result => console.log(result));
Will log:
[response1, response2, response3]
Related
In my project, have a data provider, which provides data in every 2 milli seconds. Following is the delegate method in which the data is getting.
func measurementUpdated(_ measurement: Double) {
measurements.append(measurement)
guard measurements.count >= 300 else { return }
ecgView.measurements = Array(measurements.suffix(300))
DispatchQueue.main.async {
self.ecgView.setNeedsDisplay()
}
guard measurements.count >= 50000 else { return }
let olderMeasurementsPrefix = measurements.count - 50000
measurements = Array(measurements.dropFirst(olderMeasurementsPrefix))
print("Measurement Count : \(measurements.count)")
}
What I am trying to do is that when the array has more than 50000 elements, to delete the older measurement in the first n index of Array, for which I am using the dropFirst method of Array.
But, I am getting a crash with the following message:
Fatal error: Can't form Range with upperBound < lowerBound
I think the issue due to threading, both appending and deletion might happen at the same time, since the delegate is firing in a time interval of 2 millisecond. Can you suggest me an optimized way to resolve this issue?
So to really fix this, we need to first address two of your claims:
1) You said, in effect, that measurementUpdated() would be called on the main thread (for you said both append and dropFirst would be called on main thread. You also said several times that measurementUpdated() would be called every 2ms. You do not want to be calling a method every 2ms on the main thread. You'll pile up quite a lot of them very quickly, and get many delays in their updating, as the main thread is going to have UI stuff to be doing, and that always eats up time.
So first rule: measurementUpdated() should always be called on another thread. Keep it the same thread, though.
Second rule: The entire code path from whatever collects the data to when measurementUpdated() is called must also be on a non-main thread. It can be on the thread that measurementUpdated(), but doesn't have to be.
Third rule: You do not need your UI graph to update every 2ms. The human eye cannot perceive UI change that's faster than about 150ms. Also, the device's main thread will get totally bogged down trying to re-render as frequently as every 2ms. I bet your graph UI can't even render a single pass at 2ms! So let's give your main thread a break, by only updating the graph every, say, 150ms. Measure the current time in MS and compare against the last time you updated the graph from this routine.
Fourth rule: don't change any array (or any object) in two different threads without doing a mutex lock, as they'll sometimes collide (one thread will be trying to do an operation on it while another is too). An excellent article that covers all the current swift ways of doing mutex locks is Matt Gallagher's Mutexes and closure capture in Swift. It's a great read, and has both simple and advanced solutions and their tradeoffs.
One other suggestion: You're allocating or reallocating a few arrays every 2ms. It's unnecessary, and adds undue stress on the memory pools under the hood, I'd think. I suggest not doing append and dropsFirst calls. Try rewriting such that you have a single array that holds 50,000 doubles, and never changes size. Simply change values in the array, and keep 2 indexes so that you always know where the "start" and the "end" of the data set is within the array. i.e. pretend the next array element after the last is the first array element (pretend the array loops around to the front). Then you're not churning memory at all, and it'll operate much quicker too. You can surely find Array extensions people have written to make this trivial to use. Every 150ms you can copy the data into a second pre-allocated array in the correct order for your graph UI to consume, or just pass the two indexes to your graph UI if you own your graph UI and can adjust it to accommodate.
I don't have time right now to write a code example that covers all of this (maybe someone else does), but I'll try to revisit this tomorrow. It'd actually be a lot better for you if you made a renewed stab at it yourself, and then ask us a new question (on a new StackOverflow) if you get stuck.
Update As #Smartcat correctly pointed this solution has the potential of causing memory issues if the main thread is not fast enough to consume the arrays in the same pace the worker thread produces them.
The problem seems to be caused by ecgView's measurements property: you are writing to it on the thread receiving the data, while the view tries to read from it on the main thread, and simultaneous accesses to the same data from multiple thread is (unfortunately) likely to generate race conditions.
In conclusion, you need to make sure that both reads and writes happen on the same thread, and can easily be achieved my moving the setter call within the async dispatch:
let ecgViewMeasurements = Array(measurements.suffix(300))
DispatchQueue.main.async {
self.ecgView.measurements = ecgViewMeasurements
self.ecgView.setNeedsDisplay()
}
According to what you say, I will assume the delegate is calling the measuramentUpdate method from a concurrent thread.
If that's the case, and the problem is really related to threading, this should fix your problem:
func measurementUpdated(_ measurement: Double) {
DispatchQueue(label: "MySerialQueue").async {
measurements.append(measurement)
guard measurements.count >= 300 else { return }
ecgView.measurements = Array(measurements.suffix(300))
DispatchQueue.main.async {
self.ecgView.setNeedsDisplay()
}
guard measurements.count >= 50000 else { return }
let olderMeasurementsPrefix = measurements.count - 50000
measurements = Array(measurements.dropFirst(olderMeasurementsPrefix))
print("Measurement Count : \(measurements.count)")
}
}
This will put the code in an serial queue. This way you can ensure that this block of code will run only one at a time.
I read somewhere in the past that angular.foreach is asynchronous unlike looping over arrays which is synchronous. For a long time I was taking into account this and doing the necessary to avoid executing the code which comes after the loop before it's finishes all its iterations (by wrapping the angular.foreach inside an anonymous JavaScript function which calls a callback which will be executed once the loop finishes all iterations).
(function(callback){
angular.foreach(..)
callback();
})(callback)
But I had a conversation with a collegue who didn't agree that angular.foreach is asynchronous and I also couldn't find that information again which makes me confused now.
no. Take a look at the docs
Furthermore your code wouldn't work if foreach would be asynchronous.
If foreach would be async, the callback would be called immediately after calling foreach and foreach would be put onto the eventqueue which would execute it some time in the future.
Javascripts concurrency model does not have threads but instead uses an eventloop. This means every async operation is pushed onto the eventqueue and executed later.
Have a look into the MDN
There may be a scenario where you want to make code behave asynchronously.
I had a scenario where I used local storage to store an ad-hoc user selected collection of jobs that I wanted to perform the same operation on.
I had a web service call to convert a list of job names into a returned a collection of job objects. I initially tried using a
foreach loop inside the subscribe pf the service layer, that operated on the results.
Then I tried calling another method within the foreach loop that as it performed the operations removed the job name from local storage when the operation posted to the web service correctly.
The problem was on the second iteration I read the collection of names from local storage again - before the set to remove had completed.
There was a lot of manipulation of the job and object properties to create the parameters passed on the function call, so I ended up refactoring the code, creating a value object interface and stored the information in a value object array for the whole job collection I had returned. I included the index of the job too in the value object.
I introduced a BehaviourSubject property to the class.
During the restructuring, I just added an entry to the value object array collection within the forEach loop instead. At the end of the loop. I sent next(0) to the BehaviourSubject to start the ball rolling.
Each time a job name was removed from local storage, I converted service to return a Promise.
Then in the code after the service was called I put this code in the then part, behaviour subject.next(index from value object +1)
In the initialisation I set the behaviour subject up with a -1 value..
Then in the subscription to the BehaviourSubject class I ignored -1,
And when the index +1 was > length of value object collection called completion routine - which bounce app back to prior page.
When the index was between 0 and 1 less than collection size, I just called the method that had originally been in the forEach loop with the value object entry with the value object match the index of the behaviour subject.
By doing this I had converted the behaviour of the forEach into something asynchronous.
I have an array with a few items in it. Every x seconds, I receive a new array with the latest data. I check if the data has changed, and if it has, I replace the old one with the new one:
if (currentList != responseFromHttpCall) {
currentList = responseFromHttpCall;
}
This messes up the classes provided by ng-animate, as it acts like I replaced all of the items -- well, I do actually, but I don't know how to not.
These changes can occur in the list:
There's one (or more) new item(s) in the list - not necessaryly at the end of the list though.
One (or more) items in the list might be gone (deleted).
One (or more) items might be changed.
Two (or more) items might have been swapped.
Can anyone help me in getting ng-animate to understand what classes to show? I made a small "illustation" of my problem, found here: http://plnkr.co/edit/TS401ra58dgJS18ydsG1?p=preview
Thanks a lot!
To achieve what you want, you will need to modify existing list on controller (vm.list) on every action. I have one solution that may work for your particular example.
you would need to compare 2 lists (loop through first) similar to:
vm.list.forEach((val, index)=>{
// some code to check against array that's coming from ajax call
});
in case of adding you would need to loop against other list (in your case newList):
newList.forEach((val, index)=>{
// some code to check array on controller
});
I'm not saying this is the best solution but it works and will work in your case. Keep in mind - to properly test you will need to click reset after each action since you are looking at same global original list which will persist same data throughout the app cycle since we don't change it - if you want to change it just add before end of each function:
original = angular.copy(vm.list);
You could also make this more generic and put everything on one function, but for example, here's plnkr:
http://plnkr.co/edit/sr5CHji6DbiiknlgFdNm?p=preview
Hope it helps.
UPDATE
I've tried to make a standalone version here: https://codepen.io/neezer/pen/pPRJar
It doesn't work quite like my local copy, but I'm hoping it similar enough that you can see where I'm trying to go.
I'm not getting quite the same behavior as well because I changed the listener target to document, which seemed to help some.
Also, I'm using RxJS v5 and the latest version of React.
Still getting the hang of RxJS...
I have two Observables: one subscribed to mouseover x coordinates on a table to show a resize column, and the other to allow the user to drag on that column.
Roughly speaking, the first one looks like this (all of the below defined in a componentDidUpdate lifecycle method in a React component):
Rx.DOM.mouseover(tableEl)
.map(/* some complicated x coordinate checking */)
.distinctUntilChanged()
.subscribe(/* setState call */)
That works great, and gives me this:
So now I want to provide the actual "drag" behavior, and I tried setting up a new Observable like so
// `resizerEl` is the black element that appears on hover
// from the previous observable; it's just a div that gets
// repositioned and conditionally created
Rx.DOM.mousedown(resizerEl)
.flatMap(md => {
md.preventDefault()
return Rx.DOM.mousemove(tableEl)
.map(mm => mm.x - md.x)
.takeUntil(Rx.DOM.mouseup(document))
})
.subscribe(/* do column resizing stuff */)
There are three problems with that:
Once I've done my first "drag", I can't do any more. My understanding is that takeUntil completes the Observable, and I'm not sure how I can "restart" it.
The mousemove from the first observable is still active while I'm dragging, so my black div will disappear once my x position changes enough to trigger that behavior.
The binding on the second Observable doesn't always seem to trigger (it's unreliable). I think there might be a race condition or something happening here because sometimes I'll refresh the page and I'll get the drag once (from #1), and other times I won't get it at all.
Note at first after a clean refresh I can't drag the handle (#3), then I refresh, and I can't drag the handle past the bounds setup from the first Observable--and the black resizer bar disappears and reappears as my mouse's x coordinate enters and leaves that envelope (#2).
I've been head-banging on this for quite some time now and would really appreciate any insight as to what I'm doing wrong here. In short, I want
the first Observable to "pause" when I'm dragging, then resume when I'm done dragging
the second Observable to not "complete" (or "restart") once a drag is done
the second Observable to reliably work
As I mentioned earlier, I currently have this logic setup in a React component's componentDidUpdate lifecycle method, the shape of which looks roughly like this:
componentWillUpdate() {
// bail if we don't have the ref to our table
if (!tableEl) {
return;
}
// try not to have a new Observable defined on each component update
if (!this.resizerDrag$ && this.resizer) {
this.resizerDrag$ = // second Observable from above
}
// try not to have a new Observable defined on each component update
if (!this.resizerPos$) {
this.resizerPos$ = // first Observable from above
}
}
I've played around with this a bit now, I don't think this answer will be complete, but I'd like to share my insights. Hopefully a more advanced RxJS mind will chime in, and we can all work together to figure it out :).
I recreated a "lite-er" version of this in CodePen, using some light jQuery manipulation as opposed to React. Here's what I have so far:
"the first Observable to "pause" when I'm dragging, then resume when I'm done dragging"
Solving the first point helps with the other two. Based on what I had to do to get my resizerEl, I get the feeling it is rendered in the render method of the component based on something in this.state. If this is true, that means that when the first observable still has the ability to create and destroy resizerEl even while the second observable is listening. This means that resizerEl will no longer be able to generate any events, even though the observable doesn't complete until you've moused up.
In my case, I noticed that if you moved the mouse fast enough to go outside of width of what you were trying to drag, it would eliminate resizerEl, which is of what we want, but not while we're trying to drag something!
My solution: I introduced another variable to the "state" of the "component". This would set to true when we moused down on resizerEl, and then false when we moused up again.
Then we use switchMap.
Rx.DOM.mousemove(tableEl)
.switchMap(function(event) {
return this.state.mouseIsDown ? Rx.Observable.never() : Rx.Observable.of(event);
})
.map(...
There's probably a better way to do it rather than just sticking event back in an Observable, but this was the last part of it I worked on and my brain is kind of fried hehe. The key here is switching to Observable.never while the mouse is down, that way we don't go any further down the operator chain.
Actually, one nice thing is that this may not even need to be put in this.state, since that would cause a re-render. You can probably just use an instance variable, since the variable is only essential to the Observables functionality, and not any rendering. So, using this.mouseIsDown would be just as good.
How do we handle the mouse being down or up?
Part 1:
...
Rx.DOM.mousedown(resizerEl)
.do(() => this.mouseIsDown = true)
Better to abstract this to a function of course, but this is the gist of what it does.
Part 2:
...
return Rx.DOM.mousemove(tableEl)
.map(mm => mm.x - md.x)
.takeUntil(Rx.DOM.mouseup(document))
.doOnCompleted(() => this.mouseIsDown = false)
Here we take advantage of doOnComplete to perform this side-effect once the observable has completed, which in this case, would be on mouseup.
"the second Observable to not "complete" (or "restart") once a drag is done"
Now here's the tricky one, I never ran into this problem. You see, every time Rx.DOM.mousedown(resizerEl) emits an event, inside of flatMap, a new Observable is created each time with return Rx.DOM.mousemove(tableEl).... I used RxJS 4.1 when making this, so it's possible that there could be behavioral differences, but I found that just because the inner observable completed didn't mean the outer one would complete as well.
So what could be happening? Well, I'm thinking that since you're using React, that resizerEl is being created/destroyed respectively when the component is rendering. I haven't seen the rest of your code of course, but please correct me if I'm wrong about this assumption.
This wasn't a problem for me because, for the sake of simplicity, I simply re-used the same element as a dragger, only hiding it when I wasn't hovering over a draggable element.
So the important question is: how is resizerEl being defined and used in your component? I'm assuming the actual reference to it is made using, well, a ref. But if it that DOM element is ever destroyed or recreated, then the Rx.Dom binding needs to be repeated all over again.
I see you're doing this with componentDidUpdate. However, the Rx.Dom.mousedown event may still be bound to an old copy of the ref to resizerEl. Even if the component destroys the resizer in the DOM, and sets the ref (I assume that is this.resizer) to null or undefined, that does not destroy the Observable that is bound to that element. In fact, I don't even think it removes it from memory, even if it's removed from the DOM! That means that this.resizerDrag$ will never evaluate to false/null, and it will still be listening to an element that is no longer in the DOM.
If that is the case, something like this in componentWillUpdate might help:
if (!this.resizerDrag$ && this.resizer) {
this.resizerDrag$ = // second Observable from above
}
else if (!this.resizer && this.resizerDrag$) {
this.resizerDrag$ = null;
}
This will remove the Observable if the resizer object ceases to exist, that way we can properly reinitialise it upon it's return. There's a better to way to do with Subjects, keeping the subscription to one subject and just subscribing the subject to different mousedown streams once they become available, but let's keep this simple :).
This is something where we'd have to see the rest of your code (for this component) tell what's going on, and figure how to address it. But, my hypothesis is that you'd need to intentionally destroy the Observable if this.resizer is ever removed.
the second Observable to reliably work
Pretty sure that once the above two issues work, this one goes away. Nice and easy!
The CodePen
Here is the very naive mockup I made of this problem:
https://codepen.io/anon/pen/KmapYZ
Drag the blue circles back and forth along the X axis. (Has some little problems and bugs unrelated to the scope of this question, so I'm not worried about them.)
I made some slight changes of course just to keep it in-step with the more dumb downed approach I used. But all the concepts are there, as well as most of the code you wrote, modified to match this approach.
As I mentioned before, I didn't encounter the problem of the dragging only working once, so this better demonstrates the solution to pausing the first Observable. I re-use the dragging element, which I assume is why I didn't run into the 'drag-only-once' problem.
I hope that you or anyone else can comment with some improvements to this approach, or just show us a better (potentially more idiomatic) approach.
I'm trying to use ng-grid with a sliding window of 100 records. The data is coming in realtime via signalR and every message trigger the following method:
onNewTrades(records) {
console.log("onNewRecord", records);
if (connectionStopped) return;
for (var i = 0; i < records.length; i++) {
if ($scope.recordsData.length > maxRecordsInTable)
$scope.recordsData.pop();
$scope.recordsData.unshift({
t: new Date(records[i][0]),
p: records[i][1],
a: records[i][2]
});
}
}
I have a threshold of 100 maxRecordsInTable before I start popping items off the end (before adding the new message to the front)
However, when it reaches my threshold the table simple stops updating. Strangely though, if I set a breakpoint on unshift(), the table does update with every "continue".
I suspect it's some kind of angular timing issue? I tried using $timeout()
Or may when I pop() and unshift() at the same time it doesn't pick up a change in the array? I tried using $apply() (error already in digest cycle)
There are a few things that could be happening here.
First of all, if onNewTrade is using an external, non-angular, library making xhr requests outside of angular's framework (i.e. not using $http or $resource), you have to call $scope.$apply(function(){ }) around the code you want the scope's digest to know about. That part's not clear from what you've provided. edit: Read more about when to use $scope.$apply
Second, angular's digest phase does a minimum of two passes (first to make changes, second to make sure there are no more changes). It does at most 10 passes by default. If angular evaluates the scope 10 times and it is not consistent, it gives up. see documentation. It does this because you can have multiple watch functions where one watch affects the scope higher in the hierarchy, which makes changes and affects the same watch.. basically causing an evented infinite loop. Do you see a console error about '10 $digest iterations, aborting!' or something similar?
There are a couple of other questionable things:
is onNewRecord asynchronous? If so, I would doubt connectionStopped is being done correctly. You could be returning early. Because you say a breakpoint shows values on unshift, its probably not the cause of this issue (and most likely missing $scope.$apply is the problem), but I'd rethink this code.
Your function is onNewTrade(records), but you log onNewRecord(record). If you have nested variables here, make sure you haven't excluded code that may contain typos (e.g. record instead of records). You might be working on an unexpected object.