Given I've got an array in Swift such as [1,2,3,4], a method pairs() will transform it in to the array of tuples: [(1,2), (2,3), (3,4)].
Here are some more examples of how pairs() should behave:
pairs([]) should return [] as it has no pairs.
pairs([1]) should also return [], as it has no pairs.
pairs([1,2]) should be [(1,2)]. It has just one pair.
I can write code to do this for Array, but I'd like to have pairs() available as an extension on Sequence, so that it returns a Sequence of the pairs. This would make it useable on any sequence, and compatible with methods such as map, reduce, filter, etc.
How do I go about creating a Sequence like this? And how do I write the method to transform any Sequence in this way so that it can be used as flexibly as possible?
We can use zip() and dropFirst() if we define an extension
on the Collection type:
extension Collection {
func pairs() -> AnySequence<(Element, Element)> {
return AnySequence(zip(self, self.dropFirst()))
}
}
Example:
let array = [1, 2, 3, 4]
for p in array.pairs() {
print(p)
}
Output:
(1, 2)
(2, 3)
(3, 4)
More examples:
print(Array("abc".pairs()))
// [("a", "b"), ("b", "c")]
print([1, 2, 3, 4, 5].pairs().map(+))
// [3, 5, 7, 9]
print([3, 1, 4, 1, 5, 9, 2].pairs().filter(<))
// [(1, 4), (1, 5), (5, 9)]
(Unlike I wrote in the first version of this answer ...) this
approach is not safe when applied to a Sequence, because it is
not guaranteed that a sequence can be traversed multiple times
non-destructively.
Here is a direct implementation with a custom iterator type
which works on sequences as well:
struct PairSequence<S: Sequence>: IteratorProtocol, Sequence {
var it: S.Iterator
var last: S.Element?
init(seq: S) {
it = seq.makeIterator()
last = it.next()
}
mutating func next() -> (S.Element, S.Element)? {
guard let a = last, let b = it.next() else { return nil }
last = b
return (a, b)
}
}
extension Sequence {
func pairs() -> PairSequence<Self> {
return PairSequence(seq: self)
}
}
Example:
print(Array([1, 2, 3, 4].pairs().pairs()))
// [((1, 2), (2, 3)), ((2, 3), (3, 4))]
I have RDD[(Int , Array[Double])]
eg:
1, Array(2.0,5.0,6.3)
5, Array(1.0,3.3,9.5)
1, Array(5.0,4.2,3.1)
2, Array(9.6,6.3,2.3)
1, Array(8.5,2.5,1.2)
5, Array(6.0,2.4,7.8)
2, Array(7.8,9.1,4.2)
I have to collect the distinct value of the 1st column and arrange the whole RDD according to that array.
val label_array = rdd.map(_._1).collect.distinct
Output: Array(1,5,2) and now I have to arrange data according to label_array.
Required Output
1, Array(2.0,5.0,6.3)
1, Array(5.0,4.2,3.1)
1, Array(8.5,2.5,1.2)
5, Array(1.0,3.3,9.5)
5, Array(6.0,2.4,7.8)
2, Array(9.6,6.3,2.3)
2, Array(7.8,9.1,4.2)
I have tried
val ordering = (1,5,2).productIterator.toList.zipWithIndex.toMap
rdd.sortBy{case (k,v) => ordering(k)}
But how to get the required output as the Array will be varying(elements and size difference). How can I sort RDD according to array format?
Just zipWithIndex your label_array and you should be fine
val ordering = label_array.zipWithIndex.map(x => (x._1, x._2)).toMap
And you should have your ordering map
scala.collection.immutable.Map[Int,Int] = Map(1 -> 0, 5 -> 1, 2 -> 2)
More easier way is to create a new RDD with distinct 1st column and join with the previous original column
Below is the simple example
val rdd = spark.sparkContext.parallelize(Seq(
(1, Array(2.0,5.0,6.3)),
(5, Array(1.0,3.3,9.5)),
(1, Array(5.0,4.2,3.1)),
(2, Array(9.6,6.3,2.3)),
(1, Array(8.5,2.5,1.2)),
(5, Array(6.0,2.4,7.8)),
(2, Array(7.8,9.1,4.2))
)
)
val distinct = rdd.map(v => (v._1, 1))distinct()
//(v._1, 1)this is done because you need key value to join
//now join distinct with previous original RDD
distinct.join(rdd).map(v => (v._1, v._2._2))
Output:
1, Array(2.0,5.0,6.3)
1, Array(5.0,4.2,3.1)
1, Array(8.5,2.5,1.2)
5, Array(1.0,3.3,9.5)
5, Array(6.0,2.4,7.8)
2, Array(9.6,6.3,2.3)
2, Array(7.8,9.1,4.2)
I have a 2D grid that has some obstacles in it.
[x][x][ ]
[ ][ ][ ]
[ ][ ][x]
I want to be able to find all the largest spaces of "open" grid cells.
[x][x][ ] [x][x][ ]
[1][1][ ] [2][2][2]
[1][1][x] and [ ][ ][x]
I saw the question here which is similar; however, the data I have access to is not a 2D array of values. I have a list of only the open/available indices, and this is stored in a single-dimensional array. Using the above example, my data looks like this:
[0,2]
[1,0]
[1,1]
[1,2]
[2,0]
[2,1]
I could translate it to a 2-D array, but I don't want to have to loop over all the items again to do that. I have a feeling there is some really simple solution to this, but I can't seem to find it.
Is there a way to find all the largest rectangles by only examining a list of the open indices?
EDIT
I just decided to change the array of open indices into a 2D array representing the entire grid, then use the algorithm I linked to. This works efficiently enough for my initial tests; however, I am interested to read about other possible solutions, should I need to alter my implementation for performance or space concerns.
Of course.
To find a rectangle:
Start with any free cell. Assume this is going to be the upper-left corner.
See if it can be extended horizontally.
See if it can be extended vertically.
Backtrack.
I'll ellaborate on this later, but thought this could give you a jump-start.
The list of available coordinates is sorted.
So have a global, upto-now maximum rectangle, as list too.
Do a list walk, and for every item call a function with the sublist starting at the item. To find the largest rectangle starting top-left with that item.
public List<Coord> getMaxRectangle(List<Coord> openCoords) {
List<Coord> currentMaxRectangle = null;
for (int i = 0; i < openCoords.size(); ++i) {
// Handle all rectangles starting here:
currentMaxRectangle = maxRectangleStartingAt(i, currentMaxRectangle);
}
return currentMaxRectangle;
}
public List<Coord> maxRectangleStartingAt(int i, List<Coord> openCoords) {
// Now the interesting part:
// Take the maximal width list first: standing for all widths 1, ... max width.
List<Coord> rect = new ArrayList<>();
for (int j = i; j < openCoords.size()
&& openCoords.get(j).y == openCoords.get(i).y
&& openCoords.get(j).x - openCoords.get(i).x == j - i;
++j) {
rect.add(openCoords.get(j);
}
int maxWidth = rect.size();
// Now we can look for next lines below rect.get(0), ... get(maxWidth - 1),
// continuously diminishing maxWidth till 0.
// When we need to diminish maxWidth, we can look wether the original
// maxWidth by (currentY - 1) is a maximum result.
...
}
So we walk:
+-------------+ 1st rect
| |
| | 2nd rect
| | 3rd rect
| | 4th rect
Demonstration of O(N²) complexity:
public static void main(String[] args) {
int[][] openCoords = {
{0, 2},
{1, 0},
{1, 1},
{1, 2},
{2, 0},
{2, 1}
};
new MaxRectFinder().find(openCoords);
}
private int[] maximumTopLeft;
private int[] maximumSize;
private void find(int[][] openCoords) {
maximumTopLeft = null;
maximumSize = new int[2];
for (int topLeftCandidateI = 0; topLeftCandidateI < openCoords.length; ++topLeftCandidateI) {
int yOrig = openCoords[topLeftCandidateI][0];
int xOrig = openCoords[topLeftCandidateI][1];
int y = yOrig;
int x = xOrig + 1;
int j = topLeftCandidateI + 1;
while (j < openCoords.length
&& openCoords[j][0] == y
&& openCoords[j][1] == x) {
++x;
++j;
}
// Now we have a maximum x.
for (;;) {
// Skip after right side on current line:
while (j < openCoords.length
&& openCoords[j][0] == y) {
++j;
}
++y;
// Check for maximum:
if (maximumTopLeft == null || (y - yOrig)*(x - xOrig) > maximumSize[0] * maximumSize[1]) {
maximumSize[0] = y - yOrig;
maximumSize[1] = x - xOrig;
maximumTopLeft = openCoords[topLeftCandidateI];
}
// Skip before left side below origin:
while (j < openCoords.length
&& openCoords[j][0] == y
&& openCoords[j][1] < x) {
++j;
}
if (j >= openCoords.length
|| openCoords[j][0] != y
|| openCoords[j][1] != x) {
break;
}
// Register rectangle part:
int x2 = xOrig;
while (j < openCoords.length
&& openCoords[j][0] == y
&& openCoords[j][1] == x2
&& x2 <= x) {
++x2;
++j;
}
x = x2;
}
}
if (maximumTopLeft == null) {
System.out.println("No solution found, not even 1x1.");
} else {
System.out.printf("At [%d, %d] with size [%d, %d].%n",
maximumTopLeft[0], maximumTopLeft[1], maximumSize[0], maximumSize[1]);
}
}
My previous answer was the simple approach. This a completely different one.
Algorithm
Identify horizontal runs and transform to rectangle notation.
Initialize currRects list with rectangles in row 0.
Set currRow to 1.
For each rectangle b in currRects, find intersections with horizontal runs a in row currRow. An intersect operation i is defined as
r(axi-axt,ayi-ayt) i r(bxi-bxt,byi-byt)
Its relevance is how both rectangle's y range intersect. Depicting it graphically:
bbb
aa| | No intersect
aa|aa | Partial intersect
aa|aaa| a contains b
aa|aaa|a a contains b
|aa | b contains a
|aaa| a equal b
|aaa|a a contains b
| aa| b contains a
| aa|a Partial intersect
| |aa No intersect
The necessary logic to determine each case is based in ayi, ayt, byi, and byt and simple, thus not described here.
This function will return a list of up to 2 rectangles and a flag.
No intersect ([],true)
Partial intersect: ([red,a],true)
b contains a: ([red],true)
a contains b: ([pres,a],false)
a equal b: ([pres],false)
red and pres are the result of combining the previous rectangle b with the incoming a. It can be preserving the width (in the sense of y coordinates) of the rectangle, or reducing it, but the difference is not important algorithmically.
The core of step 4 is to keep a list nextCurrRects with all the rectangles returned by function intersect. nextCurrRects should be initially empty, of course. Additionally, if for a rectangle b in the previous currRects the returned flag is never false, then b needs to be put in the final resulting list (or just printed or "yielded").
Make currRects equal to nextCurrRects. Iterate if currRow has not reached the end.
All rectangles still in currRects also belong to the result.
Example
1) inRects= [ r(0-0,2-2), r(1-1,0-2), r(2-2,0-1) ]
2) currRects= [ r(0-0,2-2) ]
3) currRow= 1
4)
nextCurrRects= []
r(1-1,0-2) i r(0-0,2-2) {a contains b} == ( [ r(0-1,2-2), r(1-1,0-2)] ], false )
nextCurrRects == [ r(0-1,2-2), r(1-1,0-2)] ]
5) currRects == [ r(0-1,2-2), r(1-1,0-2)] ]
4)
nextCurrRects= []
r(2-2,0-1) i r(0-1,2-2) {No intersect} == ( [], true )
Never false -> result= [ r(0-1,2-2) ]
r(2-2,0-1) i r(1-1,0-2) {Partial intersect} == ( r(1-2,0-1), true )
nextCurrRects= [ r(1-2,0-1) ]
Never false -> result= [ r(0-1,2-2) , r(1-2,0-1) ]
5) currRects= [ r(1-2,0-1) ]
6) result= [ r(0-1,2-2) , r(1-2,0-1) , r(1-2,0-1) ]
In fact, the algorithm just "discovered" that there is a 3rd resulting rectangle not considered in the original example:
[x][x][3]
[ ][ ][3]
[ ][ ][x]
Code
class Range:
def __init__(self,start,end=None):
self.start= start
self.end= end if end is not None else start
def isEmpty(self):
return self.start > self.end
def isUnit(self):
return self.start == self.end
def intersect(self,other):
return Range( max(self.start,other.start) , min(self.end,other.end) )
def contains(self,other):
return self.start <= other.start and self.end >= other.end
def __repr__(self):
return "Range(%d,%d)" % ( self.start, self.end )
class Rect:
def __init__(self,xRange,yRange):
self.xRange= xRange
self.yRange= yRange
def isEmpty(self):
return self.xRange.isEmpty() or self.yRange.isEmpty()
def isUnit(self):
return self.xRange.isUnit() and self.yRange.isUnit()
def intersect(self,other):
return Range( max(self.start,other.start) , min(self.end,other.end) )
def contains(self,other):
return self.xRange.contains(other.xRange) and self.yRange.contains(other.yRange)
def __repr__(self):
return "Rect(%s,%s)" % ( self.xRange, self.yRange )
def intersect(a,b):
r= Rect( Range(b.xRange.start,a.xRange.end) , a.yRange.intersect(b.yRange) )
brokenB= not a.yRange.contains(b.yRange)
fullyAbsorbedA= b.yRange.contains(a.yRange)
return (r,brokenB,fullyAbsorbedA)
def findOpenRectangles(freeElements,pastRowNum):
# From `freeElements`, compute free runs into `freeRunsPerRow`
from collections import defaultdict
freeRunsPerRow= defaultdict(set)
rowNum= -1
currRun= None
for fe in freeElements :
if fe[0] != rowNum :
if currRun is not None:
freeRunsPerRow[rowNum]|= { Rect(Range(rowNum),currRun) }
currRun= Range(fe[1],fe[1])
rowNum= fe[0]
elif fe[1] == currRun.end + 1 :
currRun= Range(currRun.start,fe[1])
else:
freeRunsPerRow[rowNum]|= { Rect(Range(rowNum),currRun) }
currRun= Range(fe[1],fe[1])
if currRun is not None:
freeRunsPerRow[rowNum]|= { Rect(Range(rowNum),currRun) }
currRun= None
for freeRuns in freeRunsPerRow.items() :
print( freeRuns )
# Yield open rectangles
currRects= set()
for currRow in range(0,pastRowNum) :
continuingRects= set()
startingRects= set( freeRunsPerRow[currRow] )
for b in currRects :
brokenB= True
for a in freeRunsPerRow[currRow] :
modifiedContinuingRect, t_brokenB, t_absorbedA= intersect(a,b)
if not modifiedContinuingRect.isEmpty() and not [ x for x in continuingRects if x.contains(modifiedContinuingRect) ] :
continuingRects-= { x for x in continuingRects if modifiedContinuingRect.contains(x) }
continuingRects|= {modifiedContinuingRect}
if not t_brokenB : brokenB= False
if t_absorbedA : startingRects-= {a}
if brokenB and not b.isUnit() : yield b
currRects= continuingRects
currRects|= startingRects
for b in currRects :
if not b.isUnit() : yield b
Test/Example
Using the following test code:
input= []
input.append(" X ")
input.append(" ")
input.append(" X ")
input.append(" X X")
input.append(" ")
input.append(" ")
input.append(" X ")
input.append(" X X ")
input.append(" X X")
# Translates input into a list of coordinates of free elements
freeElements= []
for rowNum, row in enumerate(input):
for colNum, element in enumerate(row):
if element == " " :
freeElements.append( (rowNum,colNum) )
for fe in freeElements :
print( fe )
# Find and print open rectangles
for openRect in findOpenRectangles(freeElements,len(input)) :
print( openRect )
Output is the following (some formatting to save space and titles added):
freeElements
(0, 0), (0, 1), (0, 2), (0, 4), (0, 5), (0, 6), (0, 7)
(1, 0), (1, 1), (1, 2), (1, 3), (1, 4), (1, 5), (1, 6), (1, 7)
(2, 0), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6), (2, 7)
(3, 0), (3, 1), (3, 2), (3, 3), (3, 4), (3, 6)
(4, 0), (4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6), (4, 7)
(5, 0), (5, 1), (5, 2), (5, 3), (5, 4), (5, 5), (5, 6), (5, 7)
(6, 0), (6, 1), (6, 2), (6, 3), (6, 5), (6, 6), (6, 7)
(7, 0), (7, 1), (7, 2), (7, 4), (7, 5), (7, 7)
(8, 0), (8, 2), (8, 3), (8, 4), (8, 5), (8, 6)
freeRunsPerRow
(0, {Rect(Range(0,0),Range(0,2)), Rect(Range(0,0),Range(4,7))})
(1, {Rect(Range(1,1),Range(0,7))})
(2, {Rect(Range(2,2),Range(2,7)), Rect(Range(2,2),Range(0,0))})
(3, {Rect(Range(3,3),Range(6,6)), Rect(Range(3,3),Range(0,4))})
(4, {Rect(Range(4,4),Range(0,7))})
(5, {Rect(Range(5,5),Range(0,7))})
(6, {Rect(Range(6,6),Range(5,7)), Rect(Range(6,6),Range(0,3))})
(7, {Rect(Range(7,7),Range(7,7)), Rect(Range(7,7),Range(4,5)), Rect(Range(7,7),Range(0,2))})
(8, {Rect(Range(8,8),Range(0,0)), Rect(Range(8,8),Range(2,6))})
findOpenRectangles(freeElements,len(input))
Rect(Range(0,1),Range(0,2))
Rect(Range(1,1),Range(0,7))
Rect(Range(0,2),Range(4,7))
Rect(Range(1,2),Range(2,7))
Rect(Range(3,5),Range(0,4))
Rect(Range(0,5),Range(4,4))
Rect(Range(4,5),Range(0,7))
Rect(Range(1,5),Range(2,4))
Rect(Range(1,6),Range(2,3))
Rect(Range(4,6),Range(5,7))
Rect(Range(0,6),Range(6,6))
Rect(Range(3,6),Range(0,3))
Rect(Range(3,7),Range(0,2))
Rect(Range(4,7),Range(7,7))
Rect(Range(8,8),Range(2,6))
Rect(Range(7,8),Range(4,5))
Rect(Range(0,8),Range(2,2))
Rect(Range(4,8),Range(5,5))
Rect(Range(0,8),Range(0,0))
Test/Example Conclusion
Output is correct. All non-trivial rectangles are present.