I want to get all nodes, ways and relations containing certain tags and the output file must also contain the dependent ways and nodes.
So for example I want to search for all relations with amenities and get not only the relations itself but also the dependent ways and nodes. Same for all ways with the same tags and their related nodes and finally all nodes.
Currently I found a working solution but this script takes a long time to process because it effectively reads the world map file 3 times and merges the data afterwards. I hope someone could point me to a more "straightforward" solution to increase speed.
btw. I have set the java options to "-Xmx14G -server" but the script only uses 8G of memory (the machine has 32G RAM) according to the task manager (Windows - sorry folks ;-) )
So this is the script:
set readfile=--read-pbf-fast file=planet-latest.osm.pbf workers=4
set logprogress=--log-progress interval=10
set acceptlorestags=^
place=country,state,region,province,district,county,municipality,island,islet ^
natural=sea,water,wetland,beach,coastline,marsh ^
admin_level=1,2,3,4 ^
water=* ^
wetland=*
call bin\osmosis.bat ^
%readfile% ^
--tf accept-relations ^
%acceptlorestags% ^
--used-way ^
--used-node ^
%logprogress% label="lores_rel" ^
^
%readfile% ^
--tf reject-relations ^
--tf accept-ways ^
%acceptlorestags% ^
--used-node ^
%logprogress% label="lores_way" ^
^
%readfile% ^
--tf reject-relations ^
--tf reject-ways ^
--tf accept-nodes ^
%acceptlorestags% ^
%logprogress% label="lores_node" ^
^
--merge ^
--merge ^
%logprogress% label="map_lores" ^
--mapfile-writer file=map_lores.map type=ram
One should not use a hammer for a screw. I switched to osmfilter and it works much faster now.
The full process:
convert the pbf to o5m (approx. 30 mins):
osmconvert.exe planet-latest.osm.pbf -o=planet-latest.o5m
create a filtered xml file (approx. 30 mins):
set source=planet-latest.o5m
set drop2= --drop-tags="source= fixme= created_by="
osmfilter.exe %source% --keep="natural=sea =coastline admin_level=1 =2 =3 =4 place=ocean =sea" %drop2% > map_lowres.osm
convert to mapsforge:
set osmosis=tag-conf-file=tagmapping.xml zoom-interval-conf=8,5,11,15,12,20
call bin\osmosis.bat --read-xml file=map_lowres.osm --mapfile-writer file=map_lowres.map %osmosis%
Take care that the mapfile-writer still needs days or weeks. Consider using bounding boxes and create multiple map files which is much faster than processing the whole world at once.
Related
I can't get this line to run in constant time on my micro-controller:
int zz,yy; //some binary variables
zz = (yy) ? 0 : (1 & zz);
I tried to change it to
zz = (yy) ? (0 & zz) : (1 & zz);
because & should force an evaluation of the right side even though the left side defines the result as far as I know. But it did not help.
Can anyone suggest me a solution how to make this line constant time?
It entirely depends on your compiler and its settings. Maybe (!yy) *
(1 & zz) helps?
#Quentin
I've got a set of data that I've arranged in an array within mathematica. The energy value should be as shown below when I copy as LaTex format:
Theta Phi Energy(Hartree)
1.5329 & -1.5708 & -2775.20972374594 \\
1.53476 & -1.25646 & -2775.209669993 \\
1.54014 & -0.942167 & -2775.20947403366 \\
What I'm actually getting when I copy:
1.5329 & -1.5708 & -2775.21 \\
1.53476 & -1.25646 & -2775.21 \\
1.54014 & -0.942167 & -2775.21 \\
I've done multiple data sets and have gotten it to work for all but one. There seems to be a set of 100 points that keep truncating. I've attempted the following code:
PESdatatable316 = {{"Theta", "Phi", "Energy(Hartree)"}};
Do[
PESdatatable316 = Append[PESdatatable316, {th316[[i]], phi316[[i]], NumberForm[energies316[[i]], 15]}], {i, 1, 30}]
TableForm[PESdatatable316]
where:
energies316 = Flatten[{energies100,energies216},1]
The issue seems to be within 'energies100'. The values were put in with 12 digits but truncate to 6 when I call the values.
Example:
energies100[[1]]
will output:
-2775.21
'NumberForm' corrects the values within mathematica but when I copy to LaTex form it reverts back to the truncated values.
Any ideas on how I can get these values to what they're supposed to be?
A solution has been found:
the use of 'TeXForm' with a 'NumberForm' nested inside results in the correct precision and can be copied with no issues arising.
I am trying to Validate Postal code and Phone Number in Text box using Angularjs. But it's not working
<input type="text" class="form-control errorfields" id="postalCode"
name="postalCode" ng-model="postalCode"
ng-pattern="/(^(\d{5}(-\d{4})?|[A-CEGHJ-NPRSTVXY]\d[A-CEGHJ-NPRSTV-Z]
?\d[A-CEGHJ-NPRSTV-Z]\d)$)||(^[0-9])/" required>
See this answer:
Angularjs dynamic ng-pattern validation
You can add a function from scope that returns a true or false based on the validation test. Here is some code below to check out. See the answer for additional information:
Controller:
$scope.postalCodeValidation = (function() {
var regexp = /^\(?(\d{3})\)?[ .-]?(\d{3})[ .-]?(\d{4})$/;
return {
test: function(value) {
return regexp.test(value);
}
};
})();
HTML:
<input type="text" class="form-control errorfields" id="postalCode"
name="postalCode" ng-model="postalCode"
ng-pattern="postalCodeValidation " required>
First, having double pipes in your regex syntax as mico pointed out, is invalid and will break your expression.
Second, ng-pattern can only validate one pattern per input. If you need it to validate either or, you will need to go one of two routes, create a custom directive, or add some logic to the controller to determine which expression we should check against and pass it into ng-pattern using data binding. This is bad practice in the angular world, so your best bet is to make a directive.
Edit: This will only work for regular expressions with starting ^ and ending $ metacharacters.
Like many of my StackOverflow answers, I'm super late to this party. This question got me on the right track, but none of the answers given were good enough to make a final solution.
Here's something I wrote recently after reading this thread which didn't fully answer my question, which made my code reviewer exclaim, "WTF IS THIS?!" Followed by him hitting the MERGE button.
My problem was I had a form input which I wanted to accept and validate against latitude or longitude regular expressions. I wanted to match against DMS, DDM, and DD style inputs. I could have written one giant expression, but I also needed to test the input at the component level to determine which type the user input and convert it to DD. The expressions for lat/long would make this answer even more complicated, so I am using what I think the original author intended.
If you're interested in lat/long expressions, here's a gist: https://gist.github.com/pjobson/8f44ea79d1852900457bc257a4c9fcd5
Here's kind of a rewrite of my code for this exercise, I abstracted it from angular to make it more accessible as it could be used for other purposes.
It looks like you're accepting: North America Zip Code OR UK Zip Code OR something which starts with 0-9.
Fleshing out the regex a bit, I haven't tested these myself very well, so your mileage may vary. This is what the need seemed to look like to me, it is just sample code, so it doesn't really matter that much 3+ years later.
N.America Zip or Zip+4 (from sample above)
^(\d{5}(-\d{4})?$
UK Zip Code (from wikipedia)
^([A-Za-z][A-Ha-hJ-Yj-y]?[0-9][A-Za-z0-9]? ?[0-9][A-Za-z]{2}|[Gg][Ii][Rr] ?0[Aa]{2})$
N.America Phone Number (from RegExLib.com)
^[2-9]\d{2}-\d{3}-\d{4}$
UK Phone Number (from RegExLib.com)
^(\s*\(?0\d{4}\)?\s*\d{6}\s*)|(\s*\(?0\d{3}\)?\s*\d{3}\s*\d{4}\s*)$
In an object...
const expressions = {
naZip: /^\d{5}(-\d{4})?$/,
ukZip: /^([A-Za-z][A-Ha-hJ-Yj-y]?[0-9][A-Za-z0-9]? ?[0-9][A-Za-z]{2}|[Gg][Ii][Rr] ?0[Aa]{2})$/,
naPhone: /^[2-9]\d{2}-\d{3}-\d{4}$/,
ukPhone: /^(\s*\(?0\d{4}\)?\s*\d{6}\s*)|(\s*\(?0\d{3}\)?\s*\d{3}\s*\d{4}\s*)$/
};
Now the problem is ng-pattern ONLY accepts one regex, but I have four. I want to keep those as four different things so in my component I can test to figure out which one the user input and do different actions based on whatever it is.
This looks like garbage, but it works really well.
const combinedExpression = new RegExp(`^(${_.map(expressions, exp => `(${exp.toString().replace(/^\/\^(.+?)\$\/$/,'$1')})`).join('|')})$`);
From here of course you can add this to your model and set your ng-pattern to it.
I'll try to explain this in several different ways here...
Here is a break down of what it is doing.
new RegExp(`^(${_.map(expressions, exp => `(${exp.toString().replace(/^\/\^(.+?)\$\//g,'$1')})`).join('|')})$`);
^ ^^^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^
| |||| | | | | | | | | ||
| |||| | | | | | | | | |+- end of the 1st template literal
| |||| | | | | | | | | +- end the expression with $
| |||| | | | | | | | +- join all stringified expressions with a pipe so (a|b|c|d)
| |||| | | | | | | +- end of 2nd template literal
| |||| | | | | | +- replace the like /^(\d{5}(-\d{4})?$/ to (\d{5}(-\d{4})?
| |||| | | | | +- convert the expression to a string
| |||| | | | +- new Expression interpolation
| |||| | | +- 2nd templte literal
| |||| | +- each expression is represented by exp
| |||| +- Using lodash map the expressions
| |||+- Expression interpolation
| ||+- Regex group match, we will be grouping each of the expressions
| |+- Regex starts with ^
| +- Starting a new template literal
+- Establish a new combined expression.
Further clarification...
What I'm doing is taking each of the expressions, converting them to strings, removing the starting end ending characters, joining them back together as an OR Group and then converting the whole thing to one huge combined regular expression.
Further clarification...
If that was too confusing, which I can see how it may be...
Loop through each of the expressions.
Convert each of those values to a string removing the starting and ending components so /^(\d{5}(-\d{4})?$/ converts to (\d{5}(-\d{4})?.
Join all of those together into one big OR Group so you'd get something like (a|b|c|d) where a, b, c, d are each of your stringified expressions.
Wrap the OR Group with starts with ^ and ends with $.
Wrap the entire thing with in a new RegExp.
Further clarification, in long hand JS.
// Instantiate a new empty array
const strings = [];
// Loop the expressions
for (let key in expressions) {
// Console log to see what we're getting
console.log('original expression', expressions[key]);
// assign the current expression
let stringifiedExp = expressions[key];
// convert the current expression to a string
stringifiedExp = stringifiedExp.toString();
// replace the starting /^ and ending $/ with the contents
stringifiedExp = stringifiedExp.replace(/^\/\^(.+?)\$\/$/,'$1');
// Console log to see what we got.
console.log('stringified expression', stringifiedExp);
// Push the stringified expression to strings.
strings.push(stringifiedExp);
}
// Instantiate a new string which we'll build the regex with
let expString = '^(';
expString += strings.join('|');
expString += ')$';
// Console log to see what we've got.
console.log(expString);
// Make a new Regular Expression out of the string
const combinedExpression = new RegExp(expString);
// Console log what we've got
console.log(combinedExpression);
Your regex has two || instead of single, which breaks the regex validation clause.
I pasted it to (https://regex101.com) and it says that char breaks it.
I think you need to wrap a ng-form around your input. And then you can use
[formName].postalCode.$error
I try to write an R-function which produces xtables in a loop. Later I want to call my function in a Sweave document- but a single chunk can't support multiple tables. I would have to put each table in a single chunk and wrap it with the Latex Code \begin{table} ... \end{table}.
So I wonder, whether it's possible to somehow call Sweave/knitr from within the Loop of the R-function and add \begin{table} .. \end{table} around each xtable?
Or whether it is somehow possible to send each xtable from the loop to a chunk with \begin{table} ... \end{table} environment?
A mini-example of my function:
multiple_tables_Loop<-function(...){
(....) ##Some necessary calculations to produce a data frame
for(j in 1:m){
for(i in 1:n){
a<-data.frame(...)
table<-xtable(a)
print(table)
}
}
}
In Sweave I would call the function:
<<Hallo_Table,results='aisis'>>
multiple_tables_Loop(...)
#
I'm confused by your question. xtable does include \begin{table}/\end{table} pairs. And you can put multiple tables is a code chunk (for both Sweave and knitr .Rnw files). Could it be just that you have misspelled 'asis' in your chunk header?
Showing xtable does include \begin{table}/\end{table}:
> xtable(data.frame(x=1))
% latex table generated in R 3.1.2 by xtable 1.7-4 package
% Fri Jan 23 11:12:47 2015
\begin{table}[ht]
\centering
\begin{tabular}{rr}
\hline
& x \\
\hline
1 & 1.00 \\
\hline
\end{tabular}
\end{table}
And a simple .Rnw file of
<<results="asis">>=
library("xtable")
xtable(data.frame(x=1))
xtable(data.frame(y=1))
#
properly gives two tables.
If the misspelling isn't the problem, a complete minimally reproducible example is needed along with the version numbers of R and all the packages (output of sessionInfo())
Not sure if this is the usual sort of question that gets asked around here, or if I'll get any answers to this one, but I'm looking for a pseudo-code approach to generating DB linking records from a folder structure containing image files.
I have a set of folders, structured as folllows:
+-make_1/
| +--model_1/
| +-default_version/
| | +--1999
| | +--2000
| | | +--image_01.jpg
| | | +--image_02.jpg
| | | +--image_03.jpg
| | | ...
| | +--2001
| | +--2002
| | +--2003
| | ...
| | +--2009
| +--version_1/
| | +--1999
| | ...
| | +--2009
| +--version_2/
| | +--1999
| | +--2000
| | +--2001
| | | +--image_04.jpg
| | | +--image_05.jpg
| | | +--image_06.jpg
| | | ...
| | +--2002
| | +--2003
| | | +--image_07.jpg
| | | +--image_08.jpg
| | | +--image_09.jpg
| | ...
| | +--2009
... ... ...
In essence, it represents possible images for vehicles, by year starting in 1999.
Makes and models (e.g. Make: Alfa Romeo, Model: 145) come in various trims or versions. Each trim, or version may be found in a number of vehicles which will look the same but have say differences in fuel type or engine capacity.
To save duplication, the folder structure above makes use of a default folder... And images appear for the default version from 2000 onwards. I need to produce the links table for each version - based on whether the have their own overriding images, or whether make use of the default version...
So for example, version_1 has no image files, so I need to make links for to the default images, starting in 2000 and continuing until 2009.
Version 2 on the other hand starts out using the default images in 2000, but then uses two new sets first for 2001-2002, and then 2003-2009. The list of links required are therefore...
version start end file_name
======= ===== ===== =========
version_1 2000 2009 image_01.jpg
version_1 2000 2009 image_02.jpg
version_1 2000 2009 image_03.jpg
...
version_2 2000 2001 image_01.jpg
version_2 2000 2001 image_02.jpg
version_2 2000 2001 image_03.jpg
version_2 2001 2003 image_04.jpg
version_2 2001 2003 image_05.jpg
version_2 2001 2003 image_06.jpg
version_2 2003 2009 image_07.jpg
version_2 2003 2009 image_08.jpg
version_2 2003 2009 image_09.jpg
...
(Default is just that - a place holder, and no links are required for it.)
At the moment I'm running through the folders, building arrays, and then trimming the fat at the end. I was just wondering if there was a short cut, using some sort of text-processing approach? There are about 45,000 folders, most of which are empty :-)
Here's some Python pseudocode, pretty close to executable (needs suitable imports and a def for a writerow function that will do the actual writing -- be it to an intermediate file, DB, CSV, whatever):
# first, collect all the data in a dict of dicts of lists
# first key is version, second key is year (only for non-empty years)
tree = dict()
for root, dirs, files in os.walk('make_1/model_1'):
head, tail = os.path.split(root)
if dirs:
# here, tail is a version
tree[tail] = dict
elif files:
# here, tail is a year
tree[os.path.basename(head)][tail] = files
# now specialcase default_version
default_version = tree.pop('default_version')
# determine range of years; rule is quite asymmetrical:
# for min, only years with files in them count
min_year = min(d for d in default_version if default_version[d])
# for max, all years count, even if empty
max_year = max(default_version)
for version, years in tree.iteritems():
current_files = default_version[min_year]
years.append(max_year + 1)
y = min_year
while years:
next_change = min(years)
if y < next_change:
for f in current_files:
writerow(version, y, next_change-1, f)
y = next_change
current_files = years.pop(y)
One ambiguity in the spec and example is whether it's possible for the default_version to change the set of files in some years - here, I'm assuming that doesn't happen (only specific versions change that way, the default version always carries one set of files).
If this is not the case, what happens if the default version changes in years (say) 1999 and 2003, and version1 changes in 2001 and 2005 -- what files should version 1 use for 03 and 04, the new ones in the default version, or those it specified in 01?
In the most complicated version of the spec (where both default_version and a specific one can change, with the most recent change taking precedence, and if both specific and default change in the same year then the specific taking precedence) one needs to get all the "next change year" sequence, for each specific version, by careful "priority merging" of the sequences of change years for default and specific version, instead of just using years (the sequence of changes in the specific version) as I do here -- and each change year placed in the sequence must be associated with the appropriate set of files of course.
So if the exact spec can please be expressed, down to the corner cases, I can show how to do the needed merging by modifying this pseudocode -- I'd rather not do the work until the exact specs are clarified, because, if the specs are indeed simpler, the work would be unneeded!-)
Edit: as a new comment clarified, the exact specs is indeed the most complex one, so we have do do the merging appropriately. So the loop at the end of the simplistic answer above changes to:
for version, years_dict in tree.iteritems():
# have years_dict override default_version when coincident
merged = dict(default_version, **years_dict)
current_files = merged.pop(min_year)
merged[max_year + 1] = None
y = min_year
while merged:
next_change = min(merged)
for f in current_files:
writerow(version, y, next_change-1, f)
y = next_change
current_files = merged.pop(y)
The key change is the merged = dict(... line: in Python, this means to make merged a new dict (a dict is a generic mapping, would be typically called a hashmap in other languages) which is the sum, or merge, of default_version and years_dict, but when a key is present in both of those, the value from years_dict takes precedence -- which meets the key condition for a year that's present (i.e., is a year with a change in files) in both.
After that it's plain sailing: anydict.pop(somekey) returns the value corresponding to the key (and also removes it from anydict); min(anydict) returns the minimum key in the dictionary. Note the "sentinel" idiom at merged[max_year + 1] = None: this says that the year "one after the max one" is always deemed to be a change-year (with a dummy placeholder value of None), so that the last set of rows is always written properly (with a maximum year of max_year + 1 - 1, that is, exactly max_year, as desired).
This algorithm is not maximally efficient, just simplest! We're doing min(merged) over and over, making it O(N squared) -- I think we can afford that because each merged should have a few dozen change-years at most, but a purist would wince. We can of course present an O(N logN) solution -- just sort the years once and for all and walk that sequence to get the successive values for next_change. Just for completeness...:
default_version[max_year + 1] = None
for version, years_dict in tree.iteritems():
merged = dict(default_version, **years_dict)
for next_change in sorted(merged):
if next_change > min_year:
for f in merged[y]:
writerow(version, y, next_change-1, f)
y = next_change
Here sorted gives a list with the keys of merged in sorted order, and I've switched to the for statement to walk that list from beginning to end (and an if statements to output nothing the first time through). The sentinel is now put in default_version (so it's outside the loop, for another slight optimization). It's funny to see that this optimized version (essentially because it works at a slightly higher level of abstraction) turns out to be smaller and simpler than the previous ones;-).