I have two ranges
First range("D:D") in sheet(1) starting from second row to last row, This is Lookup_values, 140.000 rows
x1 = Worksheets("1").range(Worksheets("1").Cells(2, "D").Address, Worksheets("1").Cells(Rows.Count, "D").End(xlUp)).value
Second range("A:D") in sheet(2) starting from second row to last row, this is tabble_array, 500.000 rows
x2 = Worksheets("2").range(Worksheets("2").Cells(2, 4).Address, Worksheets("2").Cells(Rows.Count, 1).End(xlUp)).value
Here i trying to look up inside arrays
ReDim ListBoxArrSplitToRows(1 To 4, 1 To UBound(x2, 1))
CX = UBound(x2, 2)
For ii = 2 To UBound(x1, 1)
For i = 1 To UBound(x2, 1)
SearchInst = x2(i, 1)
txt = x1(ii, 1)
If InStr(SearchInst, txt) Then
zz = zz + 1
For counter = 1 To 4
ListBoxArrSplitToRows(counter, zz) = x2(i, counter)
Next counter
Else
End If
Next i
Next ii
If zz <> 0 Then ReDim Preserve ListBoxArrSplitToRows(1 To 4, 1 To zz) Else ReDim ListBoxArrSplitToRows(0, 0): MsgBox "No matches"
Worksheets(1).Cells(2, "E").Resize(UBound(ListBoxArrSplitToRows, 2), 3) = ListBoxArrSplitToRows
ii=3
Ubound(x1,1) = 136586
Ubound(x2,1) = 496369
zz=1
How to quick lookup two large ranges, beacause that code takes 30 min to lookup values and it is too long
As far as I can tell, the cause of your "Subscript out of range" error is that zz is greater than UBound(x2, 1) which makes it out of ListBoxArrSplitToRows bounds.
The quick fix is to move this line
If zz <> 0 Then ReDim Preserve ListBoxArrSplitToRows(1 To 4, 1 To zz)
one line higher up, i.e. before Next i. Though get rid of Else ReDim ListBoxArrSplitToRows(0, 0); I don't know what this achieves.
Related
I have an array that can vary in size, with n columns and m rows, and I need to find all the combinations of one element for each row/column combination, but exclude any combinations where the element is zero. So, in practice, if I have:
Row
Item1
Item2
Item3
1
A
B
C
2
D
E
F
I will have 2^3 = 8 possible combinations: ABC, ABF, AEC, AEF, DBC, DBF, DEC, DEF.
But if instead of B I have a zero in row 1 Item2, I want to exclude that cell from the list of combinations (in bold above), so I would end up with: AEC, AEF, DEC and DEF.
I found some code that give me all the possible combinations on a fixed number of columns (Macro to make all possible combinations of data in various columns in excel sheet), but it doesn't account for an array that can change dimensions, or for the exclusion rule above.
I'm just going to post the code for the simple (no zeroes) case so you can see where I'm going with this (of course I have realised that Base switches over to letters for radix 11 onwards so this might not be the smartest approach :) )
Function ListCombos(r As Range)
Dim s As String, result As String
Dim arr()
Dim j As Integer, offset As Integer
Dim rows As Integer, cols As Integer
Dim nComb As Long, i As Long
rows = r.rows.Count
cols = r.Columns.Count
nComb = rows ^ cols
ReDim arr(1 To nComb)
For i = 1 To nComb
s = Application.Base(i - 1, rows, cols)
result = ""
For j = 1 To cols
offset = CInt(Mid(s, j, 1))
result = result & r.Cells(1, 1).offset(offset, j - 1)
Next j
arr(i) = result
Next i
ListCombos = arr
End Function
This is the version skipping combinations which contain zeroes. The method is to move non-zero values to the first rows of a holding array so effectively if you start with something like this
You make it look like this
So you don't have to generate or check all the combinations that contain zeroes.
Then use mixed radix to cycle through the combinations:
Option Explicit
Option Base 1
Function ListCombosWithZeroes(r As Range)
Dim s As String, result As String
Dim arr()
Dim i As Integer, j As Integer, offset As Integer, count As Integer, carry As Integer, temp As Integer
Dim rows As Integer, cols As Integer
Dim nComb As Long, iComb As Long
Dim holdingArr(20, 20) As String
Dim countArr(20) As Integer
Dim countUpArr(20) As Integer
rows = r.rows.count
cols = r.Columns.count
' Move non-zero cells to first rows of holding array and establish counts per column
For j = 1 To cols
count = 0
For i = 1 To rows
If r.Cells(i, j) <> 0 Then
count = count + 1
holdingArr(count, j) = r.Cells(i, j)
End If
Next i
countArr(j) = count
Next j
' Calculate number of combos
nComb = 1
For j = 1 To cols
nComb = nComb * countArr(j)
Next j
ReDim arr(1 To nComb)
'Loop through combos
For iComb = 1 To nComb
result = ""
For j = 1 To cols
offset = countUpArr(j)
result = result & holdingArr(offset + 1, j)
Next j
arr(iComb) = result
'Increment countup Array - this is the hard part.
j = cols
'Set carry=1 to force increment on right-hand column
carry = 1
Do
temp = countUpArr(j) + carry
countUpArr(j) = temp Mod countArr(j)
carry = temp \ countArr(j)
j = j - 1
Loop While carry > 0 And j > 0
Next iComb
ListCombosWithZeroes = arr
End Function
You don't have to have equal numbers of letters per column.
Here's a solution. Probably not most efficient, since it is O(n2), but it works.
Caveats
I put a '.' instead of zero to avoid dealing with numeric vs alphanumeric values, but you can easily change this
Since I build the strings incrementally I need indices to be predictable. Hence I fill all the possible combinations and then remove the ones containing a '.' in a second pass
Global aws As Worksheet
Global ur As Range
Global ccount, rcount, size, rptline, rptblock, iblk, iln, idx As Integer
Global tempcombos(), combos() As String
Public Sub Calc_combos()
Set aws = Application.ActiveSheet
Set ur = aws.UsedRange
ccount = ur.Columns.Count
rcount = ur.Rows.Count
size = (rcount - 1) ^ (ccount - 1)
ReDim tempcombos(size - 1)
ReDim combos(size - 1)
rptline = size / (rcount - 1)
rptblock = 1
For c = 2 To ccount
idx = 0
For iblk = 1 To rptblock
For r = 2 To rcount
For iln = 1 To rptline
tempcombos(idx) = tempcombos(idx) & Cells(r, c)
idx = idx + 1
Next iln
Next r
Next iblk
rptline = rptline / (rcount - 1)
rptblock = rptblock * (rcount - 1)
Next c
idx = 0
For iln = 0 To size - 1
If InStr(tempcombos(iln), ".") = 0 Then
combos(idx) = tempcombos(iln)
idx = idx + 1
End If
Next iln
End Sub
The Python way:
from dataclasses import dataclass, field
from itertools import product
from random import randint
from typing import Dict, List
#dataclass
class PriceComparison():
rows : int
cols : int
maxprice : int = 50
threshold : int = 0
itemcodes : List[List[str]] = field(init=False)
pricelist : Dict[str, int] = field(init=False)
def __post_init__(self):
##create sample data
self.itemcodes = [[f'A{r+self.cols*c:03d}' for c in range(self.rows)] for r in range(self.cols)]
print(self.itemcodes)
self.pricelist = {self.itemcodes[c][r]:randint(0,self.maxprice) for r in range(self.rows) for c in range(self.cols)}
##remove items with price = 0
for col in self.itemcodes:
for item in col[:]:
if self.pricelist[item] == 0:
print(f'removing {item} from {col}')
col.remove(item)
del self.pricelist[item]
def find_cheapest(self):
iterations = 1
for col in self.itemcodes:
iterations *= len(col)
print(f'this may require {iterations} iterations!')
cheapest = self.maxprice * self.cols + 1
for i, combo in enumerate(product(*self.itemcodes)):
##dummy price calculation
price = sum([self.pricelist[item] for item in combo]) * randint(1,10) // 10
if price < cheapest:
print(f'current cheapest is {price} at iteration {i}')
cheapest = price
if price < self.threshold:
print('under threshold: returning')
break
return cheapest
Some notes:
I assume the cheapest combo is not simply given by selecting the cheapest item in each column, otherwise we would not need all this complicated machinery; so I inserted a random coefficient while calculating the total price of a combo - this should be replaced with the actual formula
I also assume we have item codes in our input table, with prices for each item stored elsewhere. As sample data I create codes from 'A000' to 'Axxx', and assign a random price between 0 and a maxprice to each one
Items with price = 0 are removed immediately, before the search for the cheapest combo
For large input tables the search will take a very long time. So although it wasn't requested I also added an optional threshold parameter: if we find a total price under that value we consider it is cheap enough and stop the search
EDIT
The following is a Python 3.5 compatible version.
However it must be noted that with a 10x15 input table the number of required iterations will be somewhere near 1E+15 (something less actually, depending on how many cells we are able to ignore as "obvious outliers"). Even if we check 1 million combos per second it will still run for (something less than) 1E+09 seconds, or about 32 years.
So we need a way to improve our strategy. I integrated two options:
Setting a threshold, so that we don't search for the actual best price but stop as soon as we find an "acceptable" one
Splitting the tables in "zones" (subsets of columns), looking for the best partial solution for each zone and then combining them.
Sample runs:
##10 x 15, 5 zones, each 3 columns wide
this may require up to 1.000000e+03 iterations!
...
current best price is 1 at iteration 71 in 0.06 secs
this may require up to 1.000000e+03 iterations!
...
current best price is 2 at iteration 291 in 0.11 secs
this may require up to 1.000000e+03 iterations!
...
current best price is 1 at iteration 330 in 0.07 secs
this may require up to 8.100000e+02 iterations!
...
current best price is 4 at iteration 34 in 0.09 secs
this may require up to 1.000000e+03 iterations!
...
current best price is 1 at iteration 82 in 0.07 secs
['A000', 'A106', 'A017', 'A033', 'A139', 'A020', 'A051', 'A052', 'A008', 'A009', 'A055', 'A131', 'A147', 'A133', 'A044']
##10 x 15, no zones, threshold = 25
this may require up to 8.100000e+14 iterations!
...
current best price is 24 at iteration 267493282 in 1033.24 secs
under threshold: returning
['A000', 'A001', 'A002', 'A003', 'A004', 'A005', 'A051', 'A052', 'A008', 'A039', 'A055', 'A071', 'A042', 'A133', 'A044']
Code follows:
from itertools import product
from random import randint
from time import time
class PriceComparison():
def __init__(self, rows, cols, zones = [], maxprice = 50, threshold = 0):
self.rows = rows
self.cols = cols
if zones == []:
self.zones = [cols]
else:
self.zones = zones
self.maxprice = maxprice
self.threshold = threshold
self.__post_init__()
def __post_init__(self):
##create sample data
self.itemcodes = [['A%03d' % (r+self.cols*c) for c in range(self.rows)] for r in range(self.cols)]
print(self.itemcodes)
self.pricelist = {self.itemcodes[c][r]:randint(0,self.maxprice) for r in range(self.rows) for c in range(self.cols)}
##remove items with price = 0
for col in self.itemcodes:
for item in col[:]:
if self.pricelist[item] == 0:
print('removing %s from %s' % (item, col))
col.remove(item)
del self.pricelist[item]
def find_cheapest(self, lo, hi):
iterations = 1
for col in self.itemcodes[lo:hi]:
iterations *= len(col)
start = time()
print('\nthis may require up to %e iterations!' % (iterations))
bestprice = self.maxprice * self.cols + 1
for i, combo in enumerate(product(*self.itemcodes[lo:hi])):
##dummy price calculation
price = sum([self.pricelist[item] for item in combo]) * randint(1,10) // 10
if price < bestprice:
elapsed = time() - start
print('current best price is %d at iteration %d in %.2f secs' % (price, i, elapsed))
cheapest = combo
bestprice = price
if price < self.threshold:
print('under threshold: returning')
break
return cheapest
def find_by_zones(self):
print(self.zones)
fullcombo = []
lo = 0
for zone in self.zones:
hi = lo + zone
fullcombo += self.find_cheapest(lo, hi)
lo = hi
return fullcombo
I have a REGEX loop that finds certain strings (the Match.value in the code below). I need to preform this loop 4 times, with 4 different REGEX's, and cumulatively build the array as it finds matches in each of the four REGEX loops.
I've been able to populate a 1D array with my matches but I cannot figure out how to add other information (all static text/strings that are associated with each match) to the other dimensions. I looking to end up with something like this:
Match.value | String1_here | String2_here | String3_here | String4_here
Dim serialArray() As String
For Each Match in theMatches
Redim Preserve serialArray(x)
serialArray(x) = Match.value
x = x + 1
Next Match
Try the following...
With theMatches
If .Count > 0 Then
ReDim serialArray(1 To .Count, 1 To 5) 'allocate storage space for an N X 5 array
x = 1
For Each Match In theMatches
serialArray(x, 1) = Match.Value
serialArray(x, 2) = "String1"
serialArray(x, 3) = "String2"
serialArray(x, 4) = "String3"
serialArray(x, 5) = "String4"
x = x + 1
Next Match
End If
End With
I have code in VBA that works to generate a transition matrix on data in excel. I now have access to a huge SQL DB with ~2MM rows of information and would like to generate a transition matrix which looks something like this:
In any case here is the VBA code:
Option Explicit
Function COHORT(id, dat, rat, _
Optional classes As Integer, Optional ystart, Optional yend)
'Function is written for data sorted according to issuers and rating dates (ascending),
'rating classes numbered from 1 to classes, last rating class=default, not rated has number "0"
If IsMissing(ystart) Then ystart = Year(Application.WorksheetFunction.min(dat))
If IsMissing(yend) Then yend = Year(Application.WorksheetFunction.Max(dat)) - 1
If classes = 0 Then classes = Application.WorksheetFunction.Max(rat)
Dim obs As Long, k As Long, kn As Long, i As Integer, j As Integer, t As Integer
Dim ni() As Long, nij() As Long, pij() As Double, newrat As Integer
ReDim nij(1 To classes - 1, 0 To classes), ni(0 To classes)
obs = id.Rows.count
For k = 1 To obs
'Earliest cohort to which observation can belong is from year:
t = Application.Max(ystart, Year(dat(k)))
'Loop through cohorts to which observation k can belong
Do While t < yend
'Is there another rating from the same year?
If id(k + 1) = id(k, 1) And Year(dat(k + 1)) <= t And k <> obs Then
Exit Do
End If
'Is the issuer in default or not rated?
If rat(k) = classes Or rat(k) = 0 Then Exit Do
'Add to number of issuers in cohort
ni(rat(k)) = ni(rat(k)) + 1
'Determine rating from end of next year (=y+1)
'rating stayed constant
If id(k + 1) <> id(k) Or Year(dat(k + 1)) > t + 1 Or k = obs Then
newrat = rat(k)
'rating changed
Else
kn = k + 1
Do While Year(dat(kn + 1)) = Year(dat(kn)) And id(kn + 1) = id(kn)
If rat(kn) = classes Then Exit Do 'Default is absorbing!
kn = kn + 1
Loop
newrat = rat(kn)
End If
'Add to number of transitions
nij(rat(k), newrat) = nij(rat(k), newrat) + 1
'Exit if observation k cannot belong to cohort of y+1
If newrat <> rat(k) Then Exit Do
t = t + 1
Loop
Next k
ReDim pij(1 To classes - 1, 1 To classes + 1)
'Compute transition frequencies pij=Nij/Ni
For i = 1 To classes - 1
For j = 1 To classes
If ni(i) > 0 Then pij(i, j) = nij(i, j) / ni(i)
Next j
Next i
'NR category to the end
For i = 1 To classes - 1
If ni(i) > 0 Then pij(i, classes + 1) = nij(i, 0) / ni(i)
Next i
COHORT = pij
I am very new to SQL Server, can anyone help convert this for SQL? I made an attempt based on searching on internet but it didn't come close:
create table trans as
select a.group as from, b.group as to, count(*) as nTrans
from haveRanks as a inner join haveRanks as b
on a.ID=b.ID and a.Date+1 = b.Date
group by a.group, b.group;
create table probs as
select from, to, nTrans/sum(nTrans) as prob
from trans
group by from;
End Function
My data looks something like this:
ID Date Value
1 1/10/14 5
1 5/10/14 5
1 6/23/16 7
2 3/10/00 12
2 6/10/01 4
Edit: Answering Question from Comments:
1) Correct, if a Highest Rating is not supplied, the program will take the maximum number as the "default" class
2) If there is no rating for the year it is has not changed.
3) Not sure I understand this question, the Do While t < yend loops through the observations and checks if the next observations are in the same cohort, or if it's default/Not Rated it will kick out and go to the next one.
I'm trying to build an array for a school project, scenario is this:
You have a city that is 30 miles (y) by 20 miles (x) with roads every mile, have to write code that gives the best location for placement of a distribution center based on the business locations and cost to delivery goods to each.
I have one bit of code that records the number of client businesses, an array that records the locations of the businesses, and an array that stores volume of product each client buys.
Where I'm stuck at, is I think I should build an array that is 0 to 30, 0 to 20 (the size of the city) but I have to evaluate the cost based on user defined precision (.25, .5, 1, and 2 miles) so I should have the array be able to store the values from the calculations for 120 by 80 cells.
Not sure if that makes sense, here is the requirements:
Specifically, your program should be able to complete the following tasks:
Read the customer data, and user input for analysis resolution. Your program must be designed to
accommodate changes in the number of customers, their locations, and their product delivery volumes. User options for analysis resolution will be: 0.25 miles, 0.5 miles, 1 mile, and 2 miles.
Validate that user input is numeric and valid.
Analyze the costs at all possible distribution center locations (which may be collocated with customer
locations), and determine the optimums.
Display the X and Y values of the optimum distribution center locations and the corresponding minimum
weekly cost. There may be multiple locations with the same minimum cost.
Display the costs at all locations adjacent to the optimums, in order to show the sensitivity of the result.
The formulas to use are:
Distance to customer (Di)=abs|x-xi|+|y+yi|
Cost for customer (Ci)=1/2*Di*Vi (volume of customer product)* Ft
Ft = 0.03162*y+0.04213*x+0.4462
cost = sum Ci from 1 to number of clients
Below is my code so far, I started it to have it basically build the array and display it on a second sheet, so I can visualize it but I can't have that in the final product. In debugging it, it gets to the line of code di = and gives me a subscript out of range.
Option Explicit
Option Base 1
Sub load()
Dim Cust!, x#, y#, volume#(), n!, loc#(), i%, vol#, xi#, ci#, di#, j#
Dim choices#, val#(), nptsx!, nptsy!, Ft#, lowx!, lowy!, low!, M!
Dim costmatrix#(), npts!, cost!()
'find number of customers
Cust = 0
Do While Cells(8 + Cust, 1).Value <> ""
Cust = Cust + 1
Loop
If Cust < 2 Then
MsgBox "number of customers must be greater than 1."
Exit Sub
End If
'establist array of customer locations
ReDim loc#(1, Cust)
For j = 1 To Cust
x = Cells(7 + j, 2)
y = Cells(7 + j, 3)
Next j
ReDim volume#(Cust, 1)
For i = 1 To Cust
vol = Cells(7 + i, 4)
Next i
choices = Cells(3, 7).Value
nptsx = 30 / choices + 1
nptsy = 20 / choices + 1
'30x20 grid
ReDim costmatrix(x To nptsx, y To nptsy)
For x = 0 To nptsx - 1
Sheet3.Cells(1, x + 2) = x * choices
Next x
For y = 0 To nptsy - 1
Sheet3.Cells(2 + y, 1) = y * choices
Next y
For x = 0 To nptsx - 1
For y = 0 To nptsy - 1
For k = 1 To Cust
di = Abs(x * choices - Sheet1.Cells(7 + j, 2)) + Abs(y * choices - Sheet1.Cells(7 + j, 3))
Ft = 0.03162 * Sheet1.Cells(7 + j, 3) + 0.4213 * Sheet1.Cells(7 + j, 2) + 0.4462
ci = 1 / 2 * di * vol * Ft
Sheet3.Cells(x + 2, 2 + y) = ci
Next k
Next y
Next x
lowx = x
lowy = y
Range("I9:J:3").Clear
Range("I9") = "optimum"
Range("J9") = lowx * choices
Range("K9") = lowy * choices
Range("L9") = low
i = 9
If lowy < npts - 1 Then
i = i + 1
Cells(1, "I") = "Increment North"
Cells(1, "L") = cost(lowx, lowy + 1)
End If
If lowy > 0 Then
i = i + 1
Cells(1, "I") = "Increment South"
Cells(1, "L") = cost(lowx, lowy - 1)
End If
If lowx < npts - 1 Then
i = i + 1
Cells(1, "I") = "Increment East"
Cells(1, "L") = cost(lowx, lowy + 1)
End If
If lowx > 0 Then
i = i + 1
Cells(1, "I") = "Increment West"
Cells(1, "L") = cost(lowx, lowy - 1)
End If
End Sub
Updated, I have built the array, but I need to figure out how to do the calculations for all clients in one cell at a time, adding the results for each client together and putting the sum of them into the cell, then going onto the next cell.
When you dimension loc# via
ReDim loc#(Cust, 2)
The first index must be between 1 and Cust
The you have the loop
For k = 1 To Cust
x = Cells(7 + k, 2)
y = Cells(7 + k, 3)
Next k
After this loop runs k has value Cust + 1, not Cust since for-loops in VBA first increment the counter and then test if it has exceeded the limit.
You don't use k again until the line
di = Abs(Sheet3.Cells(1, x + 2) - loc(k, 1))
At this stage k is Cust + 1 -- which is one more than the highest permissible subscript, hence the subscript out of range error.
In context, I think that you meant to use j rather than k in both that line and the next line. I don't know if your code has other issues, but getting rid of k in those lines should help.
I just switched from VBA (Excel) to VB (Visual Studio Express 2013).
Now I have copied parts of my code from VBA to VB.
And now I'm wondering why VB is so slow...
I'm creating an Array (IFS_BV_Assy) with 4 column and about 4000 rows.
There are some identical entrys in it, so I compare every entry with each other and override the duplicate with a empty string.
The Code looks like that:
For i = 1 To counter
For y = 1 To counter
If IFS_BV_Assy(1, y) = IFS_BV_Assy(1, i) And i <> y Then
If IFS_BV_Assy(2, i) < IFS_BV_Assy(2, y) Then
IFS_BV_Assy(1, i) = ""
Else
IFS_BV_Assy(1, y) = ""
End If
Exit For
End If
Next
Next
Counter is the lenght of the Array.
In VBA it takes about 1 Sec. In VB it takes about 30 Sec. to go thru the loop. Somebody knows why? (im creating some Timestamp between every Step to be sure whats slow. And that loop is the bad guy)
The Array looks like this:
(1,1) = 12.3.015 / (2,1) = 02
(1,2) = 12.3.016 / (2,2) = 01 <-- delete
(1,3) = 12.3.016 / (2,3) = 02 <-- keep, because 02 is newer then 01
(1,4) = 12.3.017 / (2,4) = 01
(1,5) = 12.3.018 / (2,5) = 01
Thanks in advance
Andy
Edit: I create the Array like that:
strStartPath_BV_Assy = "\\xxx\xx\xx\"
myFile = Dir(strStartPath_BV_Assy & "*.*")
counter = 1
ReDim IFS_BV_Assy(0 To 2, 0 To 0)
IFS_BV_Assy(0, 0) = "Pfad"
IFS_BV_Assy(1, 0) = "Zg."
IFS_BV_Assy(2, 0) = "Rev"
Do While myFile <> ""
If UCase(Right(myFile, 3)) = "DWG" Or UCase(Right(myFile, 3)) = "PDF" Then
ReDim Preserve IFS_BV_Assy(0 To 2, 0 To counter)
IFS_BV_Assy(0, counter) = strStartPath_BV_Assy + myFile
IFS_BV_Assy(1, counter) = Left(Mid(myFile, 12), InStr(1, Mid(myFile, 12), "-") - 1)
IFS_BV_Assy(2, counter) = Mid(myFile, Len(myFile) - 8, 2)
counter = counter + 1
End If
myFile = Dir()
Loop
Maybe data was best case (around 4000) when ran in VBA.
30 sec seems a reasonable time for 4000x4000=16.000.000 iterations. 1 sec is too low for this number of iterations.
Stokke suggested to create the array as String instead of Objekt Type.
Dim IFS_BV_Assy(,) as String
I create the Module with Option Explicit Off, because I never see any difference in VBA for that point. Now I declare any variable with Dim .. as ....
And now, it's as fast as VBA is =)
Learning = making mistakes.. =)