I have an array of closures that I would like to execute and remove each item in a safe manner.
If I simply do this:
array.forEach { $0() }
array.removeAll()
It's possible that an item snuck its way in between the forEach and removeAll execution, so I might be removing an element that didn't get executed in the previous line.
Would something like this be safer?
extension Array {
mutating func removeEach(handler: #escaping (Element) -> Void) {
enumerated().forEach { handler(remove(at: $0.offset)) }
}
}
Is there a safe way to do this in an algorithmic way instead of using thread locks?
First, I assume that array here is a property (otherwise the question doesn't make a lot of sense). Second, I assume that array is already made thread-safe. Maybe something like this, which provides synchronized access at the array level.
class Container {
let arrayQueue = DispatchQueue(label: "arrayQueue", attributes: .concurrent)
var _array: [() -> Void] = []
var array: [() -> Void] {
get {
return arrayQueue.sync { return _array }
}
set {
arrayQueue.async(flags: .barrier) {
self._array = newValue
}
}
}
}
With that kind of system, you can make an atomic executor that clears the array:
func execute() {
arrayQueue.async(flags: .barrier) {
// Make a local copy of the array
let toExecute = self._array
// Clear the array (we have a barrier, so there's no race condition)
self._array.removeAll()
// async to another queue to avoid blocking access while executing
DispatchQueue.global().async {
for closure in toExecute {
closure()
}
}
}
}
(You mentioned "without using thread locks." You should almost never be using thread locks in Swift. That's what GCD is for, and it does a lot of work to avoid locking so you don't have to. If your goal were to do this without using GCD, then no, there is no way to safely mutate a property from multiple threads without a concurrency system of some sort, and the best concurrency system for Swift is GCD.)
Related
HI there I am writing a function to get data from my parse server in swift everything is working alright and the data is getting read well. but when I try to return the array it returns me an empty array.
I also added a print in the "Get data in background" and there, the array was getting full.So the problem is not getting the data.
public func getthearray() -> Array<UIImage>
{
let user = PFUser.current()
let array = user?["Photos"] as! Array<PFFileObject>
var imagearray = [UIImage]()
for x in array
{
x.getDataInBackground
{ (dataa, error) in
var img = UIImage(data: dataa!)
imagearray.append(img!)
}
}
return imagearray
}
The reason the function returns an empty array is the fact that the code in the x.getDataInBackground completion gets called asynchronously, and before any image will get added to the array, the faction will already return.
Here is the solution (in 2 versions), which is your function slightly refactored and rewritten with completion handler and Grand Central Dispatch
Approach 1. Semaphores
public func getTheArray(_ completion: #escaping ([UIImage]) -> Void) {
let user: PFUser = .current()
let array = user?["Photos"] as? [PFFileObject] ?? []
var result = [UIImage]()
let semaphore = DispatchSemaphore(value: 0)
// dispatch to a separate thread which we can safely occupy and block
// while waiting for the results from async calls.
DispatchQueue.global().async {
for file in array {
file.getDataInBackground { (data, error) in
if let data = data, let image = UIImage(data: data) {
result.append(image)
}
// the result have arrived, signal that we are ready
// to move on to another x
semaphore.signal()
}
// wait until the `signal()` is called, thread is blocked
semaphore.wait()
}
// dispatch to main queue where we are allowed to perform UI updates
DispatchQueue.main.async {
// call the completion, but we are doing it only once,
// when the function is finished with its work
completion(result)
}
}
}
Approach 2. Dispatch Groups
public func getTheArray(_ completion: #escaping ([UIImage]) -> Void) {
let user: PFUser = .current()
let array = user?["Photos"] as? [PFFileObject] ?? []
var result = [UIImage]()
let group = DispatchGroup()
for file in array {
group.enter()
file.getDataInBackground { (data, error) in
if let data = data, let image = UIImage(data: data) {
result.append(image)
}
group.leave()
}
}
// notify once all task finish with `leave()` call.
group.notify(queue: .main) {
// call the completion, but we are doing it only once,
// when the function is finished with its work
completion(result)
}
}
Usage
and you would call it like this
getTheArray { result in
// do what you want with result which is your array of UIImages
}
Documentation:
DispatchQueue
DispatchSemaphore
Relevant blogposts:
Grand Central Dispatch by John Sundell
A deep dive into Grand Central Dispatch in Swift by John Sundell
You should use the completion handler closure as discussed by others, in order to asynchronously return the array of images when all the requests are done.
The general pattern is:
Use #escaping completion handler closure.
Use dispatch group to keep track of all of the asynchronous requests.
Store the results in a dictionary as the individual requests finish. We use a dictionary for the results, because with concurrent requests, you do not know in what order the requests will finish, so we will store it in a structure from which we can efficiently retrieve the results later.
Use dispatch group notify closure, to specify what should happen when all the requests are done. In this case, we will build a sorted array of images from our unsorted dictionary, and call the completion handler closure.
As an aside, one should avoid using forced unwrapping operator (the ! or as!), especially when dealing with network requests, whose success or failure is outside of your control. We would generally use guard statement to test that the optionals were safely unwrapped.
Thus:
func fetchImages(completion: #escaping ([UIImage]) -> Void) {
guard
let user = PFUser.current(),
let files = user["Photos"] as? [PFFileObject]
else {
completion([])
return
}
var images: [Int: UIImage] = [:] // dictionary is an order-independent structure for storing the results
let group = DispatchGroup() // dispatch group to keep track of asynchronous requests
for (index, file) in files.enumerated() {
group.enter() // enter the group before asynchronous call
file.getDataInBackground { data, error in
defer { group.leave() } // leave the group when this completion handler finishes
guard
let data = data,
let image = UIImage(data: data)
else {
return
}
images[index] = image
}
}
// when all the asynchronous tasks are done, this `notify` closure will be called
group.notify(queue: .main) {
let array = files.indices.compactMap { images[$0] } // now rebuild the ordered array
completion(array)
}
}
And, you'd use it like so:
fetchImages { images in
// use `images` here, e.g. if updating a model object and refreshing the UI, perhaps:
self.images = images
self.tableView.reloadData()
}
// but not here, because the above runs asynchronously (i.e. later)
But the idea is to embrace asynchronous patterns given that we are calling an asynchronous API, and use dispatch groups to keep track of when they are all done. But dispatch semaphores are generally discouraged, as they force the requests to be performed sequentially, one after another, which will slow it down (e.g., in my experience, between two and five times slower).
As an aside, we would generally want to report success or failure. So rather than [UIImage], we would have the closure’s parameter to be a Result<[UIImage], Error> (if you want a single Error) or [Result<UIImage, Error>] if you want either a UIImage or Error for each file. But there’s not enough here to know what error handling you want. But just a FYI.
While I've answered the tactical question above, we really should ask how many images might you be dealing with. If you are retrieving lots of images and/or the user is on a slow connection, this pattern (known as “eager” fetching of the images) is discouraged. First, it can be slow if there are a lot of them. Second, images take up a lot of memory, and you might not want to load all of them if you only need a subset of them at any given point in time. We would often refactor our app to employ “lazy” fetching of the images, retrieving them as needed, not all up front. Or, if you want to do eager fetch, download the assets as files to caches folder (reducing RAM consumption) and create UIImage instances when required in the UI.
I am taking some inspiration from
https://marcosantadev.com/swift-arrays-holding-elements-weak-references/
and I want to be able to maintain an array holding weak references to its elements, so that in case those elements get released elsewhere in my code base, I don't retain them in my array.
I would like the implementation to be as type safe as possible, however should be reusable.
The strategy that I am using is declaring a Weak Reference container as so.
class WeakRefContainer<T> where T: AnyObject {
private(set) weak var value: T?
init(value: T?) {
self.value = value
}
}
Then I want to maintain an array of these WeakRefContainers, so I create an array extension:
extension Array where Element: WeakRefContainer<AnyObject> {
func compact() -> [WeakRefContainer<AnyObject>] {
return filter { $0.value != nil }
}
}
When calling the compact method, I am now able to clear up the array in case stuff needs to be cleaned up.
I am now having some compilation issues which am having trouble understanding.
Lets suppose I have a sample class
class SampleClass {
}
And I try to use everything as follows:
var weakReferencesArray = [WeakRefContainer<SampleClass>]()
let obj1 = WeakRefContainer.init(value: SampleClass())
let obj2 = WeakRefContainer.init(value: SampleClass())
weakReferencesArray.append(obj1)
weakReferencesArray.append(obj2)
weakReferencesArray.compact()
When I try to call compact I get the following error message:
MyPlayground.playground:29:21: 'WeakRefContainer<SampleClass>' is not a subtype of 'WeakRefContainer<AnyObject>'
Can anyone unblock me please? Thanks
Your code doesn't work because WeakRefContainer<SampleClass> is not a subclass of WeakRefContainer<AnyObject> because generics are invariant in Swift. Thus weakReferencesArray can't use the compact method added from the extension.
There is a workaround for this, via a protocol:
protocol WeakHolder {
var hasRef: Bool { get }
}
extension WeakRefContainer: WeakHolder {
var hasRef: Bool { return value != nil }
}
extension Array where Element: WeakHolder {
func compacted() -> [Element] {
return filter { $0.hasRef }
}
mutating func compact() {
self = compacted()
}
}
I also renamed compact to compacted, for better Swift semantics, and replaced the original compact by a mutating version.
You probably want the extension to apply to all [WeakRefContainer<T>] where T can be any type extending AnyObject.
extension Array where Element: WeakRefContainer<T> {
However, currently, parameterised extensions are not possible. See this proposal.
You can kind of work around this by making compact generic:
extension Array{
func compact<T>() -> [Element] where Element == WeakRefContainer<T> {
return filter { $0.value != nil }
}
}
I've just read a post by Basem Emara about creating a threadsafe array Type in Swift. While I glanced through the code example, I asked myself if there isn't a way to achieve this with quite less code.
Suppose I create this class:
// MARK: Class Declaration
class ThreadsafeArray<Element> {
// Private Variables
private var __array: [Element] = []
private var __arrayQueue: DispatchQueue = DispatchQueue(
label: "ThreadsafeArray.__concurrentArrayQueue",
attributes: .concurrent
)
}
// MARK: Interface
extension ThreadSafeArray {
// ReadWrite Variables
var threadsafe: [Element] {
get {
return self.__arrayQueue.sync {
return self.__array
}
}
set(newArray) {
self.__arrayQueue.async(flags: .barrier) {
self.__array = newArray
}
}
}
}
If, from now on, I only accessed the actual array through .threadsafe, would this suffice to make the array threadsafe?
Also, could I implement it a struct instead of a class to get the mutating checks as well?
I am aware that the objects inside this array would not be threadsafe themselves through this but this is not the point, so let's assume I only put threadsafe stuff in there.
(Of course, to avoid the calls to .threadsafe, I would make the shiny new class conform to ExpressibleByArrayLiteral, Collection and RangeReplaceableCollection, so I can use it like a normal array.
Edit
Meanwhile, I've tried testing it in a playground and have come to believe that it doesn't suffice.
Playground code:
import Foundation
import PlaygroundSupport
PlaygroundPage.current.needsIndefiniteExecution = true
// Testing //
// Thread-unsafe array
func unsafeArray() {
var array: [Int] = []
var iterations: Int = 1000
let start: TimeInterval = Date().timeIntervalSince1970
DispatchQueue.concurrentPerform(iterations: iterations) { index in
let last: Int = array.last ?? 0
array.append(last + 1)
DispatchQueue.global().sync {
iterations -= 1
// Final loop
guard iterations <= 0 else { return }
print(String(
format: "Unsafe loop took %.3f seconds, count: %d.",
Date().timeIntervalSince1970 - start, array.count
))
}
}
}
// Thread-safe array
func safeArray() {
let array: ThreadsafeArray<Int> = ThreadsafeArray<Int>()
var iterations: Int = 1000
let start: TimeInterval = Date().timeIntervalSince1970
DispatchQueue.concurrentPerform(iterations: iterations) { index in
let last: Int = array.threadsafe.last ?? 0
array.threadsafe.append(last + 1)
DispatchQueue.global().sync {
iterations -= 1
// Final loop
guard iterations <= 0 else { return }
print(String(
format: "Safe loop took %.3f seconds, count: %d.",
Date().timeIntervalSince1970 - start, array.threadsafe.count
))
}
}
}
unsafeArray()
safeArray()
Output:
Most of the time:
experiments(31117,0x7000038d0000) malloc: *** error for object 0x11f663d28: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Sometimes:
IndexError: Index out of range
Unfortunately also:
Unsafe loop took 1.916 seconds, count: 994.
Safe loop took 11.258 seconds, count: 515.
Doesn't seem to suffice (also, it's incredibly unperformant).
The synchronization mechanism in your question, with concurrent queue and judicious use of barrier is known as the “reader-writer” pattern. In short, it offers concurrent synchronous reads and non-concurrent asynchronous writes. This is a fine synchronization mechanism. It is not the problem here.
But there are a few problems:
In the attempt to pare back the implementation, this class has become very inefficient. Consider:
class ThreadSafeArray<Element> {
private var array: [Element]
private let queue = DispatchQueue(label: "ThreadsafeArray.reader-writer", attributes: .concurrent)
init(_ array: [Element] = []) {
self.array = array
}
}
extension ThreadSafeArray {
var threadsafe: [Element] {
get { queue.sync { array } }
set { queue.async(flags: .barrier) { self.array = newValue } }
}
}
let numbers = ThreadSafeArray([1, 2, 3])
numbers.threadsafe[1] = 42 // !!!
What that numbers.threadsafe[1] = 42 line is really doing is as follows:
Fetching the whole array;
Changing the second item in a copy of the array; and
Replacing the whole array with a copy of the array that was just created.
That is obviously very inefficient.
The intuitive solution is to add an efficient subscript operator in the implementation:
extension ThreadSafeArray {
typealias Index = Int
subscript(index: Index) -> Element {
get { queue.sync { array[index] } }
set { queue.async(flags: .barrier) { self.array[index] = newValue} }
}
}
Then you can do:
numbers[1] = 42
That will perform a synchronized update of the existing array “in place”, without needing to copy the array at all. In short, it is an efficient, thread-safe mechanism.
What will end up happening, as one adds more and more basic “array” functionality (e.g., especially mutable methods such as the removing of items, adding items, etc.), you end up with an implementation not dissimilar to the original implementation you found online. This is why that article you referenced implemented all of those methods: It exposes array-like functionality, but offering an efficient and (seemingly) thread-safe interface.
While the above addresses the data races, there is a deep problem in that code sample you found online, as illuminated by your thread-safety test.
To illustrate this, let’s first assume we flesh out our ThreadSafeArray to have last, append() and make it print-able:
class ThreadSafeArray<Element> {
private var array: [Element]
private let queue = DispatchQueue(label: "ThreadsafeArray.reader-writer", attributes: .concurrent)
init(_ array: [Element] = []) {
self.array = array
}
}
extension ThreadSafeArray {
typealias Index = Int
subscript(index: Index) -> Element {
get { queue.sync { array[index] } }
set { queue.async(flags: .barrier) { self.array[index] = newValue} }
}
var last: Element? {
queue.sync { array.last }
}
func append(_ newElement: Element) {
queue.async(flags: .barrier) {
self.array.append(newElement)
}
}
}
extension ThreadSafeArray: CustomStringConvertible {
var description: String {
queue.sync { array.description }
}
}
That implementation (a simplified version of the rendition found on that web site) looks OK, as it solves the data race and avoids unnecessary copying of the array. But it has its own problems. Consider this rendition of your thread-safety test:
let numbers = ThreadSafeArray([0])
DispatchQueue.concurrentPerform(iterations: 1_000) { <#Int#> in
let lastValue = numbers.last! + 1
numbers.append(lastValue)
}
print(numbers) // !!!
The strict data race is solved, but the result will not be [0, 1, 2, ..., 1000]. The problem are the lines:
let lastValue = numbers.last! + 1
numbers.append(lastValue)
That does a synchronized retrieval of last followed by a separate synchronized append. The problem is that another thread might slip in between these two synchronized calls and fetch the same last value! You need to wrap the whole “fetch last value, increment it, and append this new value” in a single, synchronized task.
To solve this, we would often give the thread-safe object a method that would provide a way to perform multiple statements as a single, synchronized, task. E.g.:
extension ThreadSafeArray {
func synchronized(block: #escaping (inout [Element]) -> Void) {
queue.async(flags: .barrier) { [self] in
block(&array)
}
}
}
Then you can do:
let numbers = ThreadSafeArray([0])
DispatchQueue.concurrentPerform(iterations: 1_000) { <#Int#> in
numbers.synchronized { array in
let lastValue = array.last! + 1
array.append(lastValue)
}
}
print(numbers) // OK
So let’s return to your intuition that the author’s class can be simplified. You are right, that it can and should be simplified. But my rationale is slightly different than yours.
The complexity of the implementation is not my concern. It actually is an interesting pedagogical exercise to understand barriers and the broader reader-writer pattern.
My concern is that (to my point 3, above), is that the author’s implementation lulls an application developer in a false sense of security provided by the low-level thread-safety. As your tests demonstrate, a higher-level level of synchronization is almost always needed.
In short, I would stick to a very basic implementation, one that exposes the appropriate high-level, thread-safe interface, not a method-by-method and property-by-property interface to the underlying array, which almost always will be insufficient. In fact, this desire for a high-level, thread-safe interface is a motivating idea behind a more modern thread-safety mechanism, namely actors in Swift concurrency.
I suspect this line is your issue:
DispatchQueue.global().sync { ...
If you specify one serial queue you want to use here you should get the result you want.
Something like:
let array = SynchronizedArray<Int>()
var iterations = 1000
let queue = DispatchQueue(label: "queue")
DispatchQueue.concurrentPerform(iterations: 1000) { index in
array.append(array.last ?? 0)
queue.sync {
iterations -= 1
if iterations == 0 {
print(array.count)
}
}
}
Another method of locking objects is:
func lock(obj: AnyObject, work:() -> ()) {
objc_sync_enter(obj)
work()
objc_sync_exit(obj)
}
Could your class use this to lock its standard array when needed?
I am trying to make an array of integers with data I have stored on an online database (Parse). I know for a fact that I am receiving the data and that my variable is storing that data. I also know for a fact that the data value is being added as when I print out the size of the array under my append line, the size constantly increases until it reaches the size of my database (90). However, when I print the size of the array at the end of the method or in my constructor, I get a value of 0. I have a feeling that this problem is happening because of the "self" keyword but I am not sure how to get around it. My end goal is store all the values from the database into an array that I create and can globally access.
Here is the code:
import Foundation
class DataLoader
{
var allData: [Int] = []
init()
{
allData = []
generateAllData()
println(allData.count)
}
private func generateAllData()
{
var tempVal: Int = 0
var tempArray: [Int] = []
Parse.setApplicationId("CENSORED FOR PRIVACY REASONS", clientKey: "CENSORED FOR PRIVACY REASONS")
var query = PFQuery(className: "Snapshot")
query.selectKeys(["objectID"])
query.findObjectsInBackgroundWithBlock
{
(objects: [AnyObject]!, error: NSError!) -> Void in
if error == nil
{
for obj in objects
{
var temp: String = obj.objectId
var newQuery = PFQuery(className: "Data")
newQuery.getObjectInBackgroundWithId(temp)
{
(dataValue: PFObject!, error: NSError!) -> Void in
if error == nil && dataValue != nil
{
tempVal = dataValue["Year"].integerValue
}
else
{
println(error)
}
}
self.allData.append(tempVal)
println(self.allData.count)
}
}
}
println(allData.count)
}
}
The problem is that the println's at the end of the method and the end of the init are happening while the background operation is still getting the data.
When you call "query.findObjectsInBackgroundWithBlock", it will go off into the background and get the data. But that does not block the next line from being called and it does not stop the function returning.
I suggest you add a callback or delegate method so that the rest of your code knows when the data has completely downloaded.
And you use of "self" is correct. Inside a block, you have to provide that context.
Update:
Adding more info about blocks and callbacks.
When you are using a block, the code after the block continues on without waiting for the block to complete. So a good way to know when the block has finished it to send a callback function as one of the parameters to the function containing the block. Once the block has completed, it can use the callback function to report back.
As a shorter example than your code, have a look at this animation function:
doSomeAnimation() // call the function below
func doSomeAnimation() {
println("Starting doSomeAnimation")
UIView.animateWithDuration(2.0,
animations: { () -> Void in
// animate something here
self.alphaButton.alpha = 1.0
}) { (animationDone) -> Void in
// animation over
println("Animation complete")
}
println("Ending doSomeAnimation")
}
You will see it contains 3 println() statements and the order in which they appear may surprise you:
Starting doSomeAnimation
Ending doSomeAnimation
Animation complete
So the doSomeAnimation() function is over long before the animation has completed. In this trivial example, that doesn't matter, but what if you need to know before the rest of your code could proceed?
One solution is to send a callback function to the function with the block. Here is the previous example with a callback in place:
doSomeAnimation() { (complete : Bool) in
println("Animation callback")
}
func doSomeAnimation(callback : ( Bool ) -> Void) {
println("Starting doSomeAnimation")
UIView.animateWithDuration(2.0,
animations: { () -> Void in
// animate something here
self.alphaButton.alpha = 1.0
}) { (animationDone) -> Void in
// animation over
println("Animation complete")
callback(animationDone)
}
println("Ending doSomeAnimation")
}
So the doSomeAnimation() function is called, but it is provided with a function as its parameter. It performs the animation and when the animation has finished, it uses this supplied callback function to report back to the caller.
And now the println() statements show as follows:
Starting doSomeAnimation
Ending doSomeAnimation
Animation complete
Animation callback
Hopefully this makes it clearer, both what the issue is an a possible solution.
First: I'm really new at CloudKit and the "dispatch"-mechanisms.
But I'd like to write a function in swift which will return the query-results as an array. Now my problem is, that the function doesn't wait for the query to finish and so I receive an empty array. There are records in my DB but I don't get it back properly.
So as I said I don't really understand the whole dispatch_async-mechanism etc. I've also read the tutorial on raywenderlich.com about CloudKit but I still don't get, how I can return the array properly.
This is my actual code. I've often seen people using dispatch_async-methods but I really don't understand how they can return my array.
func loadMyShopsCoolAwesome() ->[Shops]{
let container = CKContainer.defaultContainer()
var publicDB = container.publicCloudDatabase
let myQuery = CKQuery(recordType: "Shops", predicate: NSPredicate(value: true))
var myShops = [MyShops]()
publicDB.performQuery(myQuery, inZoneWithID: nil) {
results, error in
if error != nil {
println(error)
} else {
for record in results{
let shop = MyShops(nameElementAt: record.objectForKey("nameElementAt") as Int, nameElementFromSplit: record.objectForKey("nameElementFromSplit") as Int, nameSplitString: record.objectForKey("nameSplitString"), priceElementAt: record.objectForKey("priceElementAt") as Int, priceSplitString: record.objectForKey("priceSplitString"), shopName: record.objectForKey("shopName"), shopURL: record.objectForKey("shopURL"), xPathName: record.objectForKey("xPathName"), xPathPrice: record.objectForKey("xPathPrice"))
myShops.append(shop)
}
return myShops
}
}
}
You should not let your function return data. This way it will only work if you implement a wait mechanism (usually done with semaphores). If you would do that, you would block your app during the data fetch.
Instead you should return void and instead of returning your shops you should call a new function with the shops. There you should continue processing of the shops. So your function will be:
func loadMyShopsCoolAwesome() {
...
And instead of the
return myShops
You should call a new function. something like:
processShops(myShops)
Which would then call a new function that looks like:
func processShops(shops: [Shops]) {
// do work with shops
...
}
Also be aware that this will be called on a background queue. So if you want to do something with the interface you have call that on the main queue like this:
NSOperationQueue.mainQueue().addOperationWithBlock {
...
}