Why Server Vector Tiles for MapboxGL Client rendered incorrectly - postgis

I am trying to set up a web server in Node.js that serves vector tiles to be displayed in a browser using MapboxGL JS. The data for the vector tiles is stored in a PostGIS database.
I receive a geojson file and process it createing a database with next sentence: CREATE TABLE IF NOT EXISTS countries ( table_id SERIAL, properties jsonb not null, geom geometry(GeometryZ,4326), primary key (table_id));
My current set up seems to going in the right direction, as I can see vector tiles being loaded and displayed in the browser. However the rendered result is incorrect (this is a screenshot of a section of my map):
What is rendering incorrectly???
Here is the code:
var express = require('express');
var SphericalMercator = require('sphericalmercator');
var mercator = new SphericalMercator({
size: 256 //tile size
});
const { pool } = require('../postgressql/config');
var app = express();
app.get('/:namelayer/:z/:x/:y.pbf', (req, res, next) => {
var options = {
x: parseInt(req.params.x),
y: parseInt(req.params.y),
z: parseInt(req.params.z),
layerName: req.params.namelayer
};
const bbox = mercator.bbox(options.x, options.y, options.z, false, '3857');
const sql = `
SELECT ST_AsMVT(q, '${options.layerName}', 4096, 'geom')
FROM (
SELECT
table_id, properties,
ST_AsMVTGeom(
geom,
ST_MakeEnvelope(${bbox[0]}, ${bbox[1]}, ${bbox[2]}, ${bbox[3]}, 4326),
4096,
256,
false
) AS geom
FROM ${'public.'+options.layerName} c
) q;
`;
try {
pool.query(sql, (error, results) => {
if (error) {
return res.status(500).json({
ok: false,
message: error
});
}
const tile = results.rows[0];
// set the response header content type
res.setHeader('Content-Type', 'application/x-protobuf');
// trigger catch if the vector tile has no data, (return a 204)
if (tile.st_asmvt.length === 0) {
res.status(204);
}
// send the tile!
res.send(tile.st_asmvt);
});
} catch (e) {
res.status(404).send({
error: e.toString(),
});
}
});
module.exports = app;
I followed using the idea from this sites:
https://medium.com/nyc-planning-digital/using-the-new-mvt-function-in-postgis-75f8addc1d68
https://geoinquiets.github.io/taller-vt/8_postgis_vt/
https://blog.jawg.io/how-to-make-mvt-with-postgis/

Evidently, you have a projection problem. It looks like the black overlay is more stretched vertically towards the poles than the basemap, as well as being shifted northwards.
It's a bit hard to say for sure where the problem lies, as you haven't includede the client side code. It's possible the vector tiles are perfectly represented within the EPSG:3857 projection, but you're overlaying them on a slightly different projection (EPSG:3785 perhaps).
The fact that you've disabled ST_AsMVTGeoms clip_geom parameter also suggests that your bounding box wasn't calculated correctly, and you were working around that.
You may get better answers posting the question at gis.stackexchange.com

Related

How to convert HEIC to JPG in React?

HEIC is Apple's own format to store high resolution images made with iOS cameras. But I would store on backend JPG, because HEIC is not displayed in most browsers, not even in Safari.
SOLUTION 1
I tried this for the conversion:
const buffer = Buffer.from(await file.arrayBuffer())
const d = heicConvert({ buffer, format: 'JPEG' })
const imgBase64 = btoa(
d.reduce((data, byte) => `${data}${String.fromCharCode(byte)}`, '')
)
but because I use Next.js it is not compatible with it.
Failed to compile.
./node_modules/libheif-js/libheif/libheif.js
Module not found: Can't resolve 'fs' in '/Users/janoskukoda/Workspace/tikex/portal/team/node_modules/libheif-js/libheif'
SOLUTION 2
I tried this also:
export default uploadImage
const buffer = await file.arrayBuffer()
const image = sharp(buffer)
const metadata = await image.metadata()
if (metadata.format === 'heic') {
// Convert the image to JPG
const jpgBuffer = await image.jpeg().toBuffer()
// Encode the JPG image as a base64 string
const imgBase64 = btoa(
jpgBuffer.reduce((data, byte) => `${data}${String.fromCharCode(byte)}`, '')
)
}
But I can not compile, seems sharp is not recommended to use in client side.
Do you have any other way to do it?
Anyway idea comes here: https://itnext.io/tackling-iphone-or-ipad-images-support-in-browser-8e3e64e9aaa1
If you show me a solution uses serverless api, it is also ok. It is important file comes from html input element.
SOLUTION 3
import loadImage from 'blueimp-load-image'
convertedImage = await new Promise((resolve, reject) => {
loadImage(
propsAndFile.file,
resolve,
{ orientation: true, canvas: true },
reject
)
})
Before getting to the answer: You can never trust data uploaded by a client — you must always validate/convert using a process that is not accessible by a user (a backend process) to ensure data validity: even if there are mechanisms in place to validate received network requests as coming from an authenticated user on your site, any user can still use developer tools to execute arbitrary JavaScript and send whatever kinds of network requests with whatever payloads that they want to.
Regarding iOS devices and getting JPEG instead of HEIF from a file input: You don't need to do this yourself — iOS can do it for you.
<input type="file"> elements support an accept attribute that can be used to restrict the kinds of file media types that can be uploaded: read more at the Limiting accepted file types section of the MDN documentation article for <input type="file">.
Below is an example which shows how to use that attribute. When an iOS user selects the input, they can choose to take a photo using their camera or select one from the photo library. iOS will perform the necessary file conversion to JPEG automatically in both of these cases (for example, even when a selected image from the photo library is in HEIF format). The example demonstrates this when you try it with an HEIF-encoded image on an iOS device:
const input = document.getElementById('image-upload');
const output = document.getElementById('output');
const updateDisplayedFileInfo = () => {
const file = input.files?.[0];
if (!file) {
output.textContent = 'No file selected';
return;
}
const dateModified = new Date(file.lastModified).toISOString();
const {name, size, type} = file;
const data = {
dateModified,
name,
size,
type,
};
const json = JSON.stringify(data, null, 2);
output.textContent = json;
};
input.addEventListener('change', updateDisplayedFileInfo);
updateDisplayedFileInfo();
pre { background-color: hsla(0, 0%, 50%, 0.1); padding: 0.5rem; } code { font-family: monospace; }
<input id="image-upload" type="file" accept="image/jpeg" />
<pre><code id="output"></code></pre>
Install the heic2jpeg library by running the following command in your terminal:
npm install heic2jpeg
Import the heic2jpeg library in your React component:
import heic2jpeg from 'heic2jpeg';
Convert the HEIC file to JPG by calling the convert method of the heic2jpeg library and passing in the HEIC file as an argument:
const jpegData = await heic2jpeg.convert(heicFile);
You can then use the jpegData to create a new JPG file or display it in an img element:
const jpegFile = new File([jpegData], 'image.jpg', { type: 'image/jpeg' });
// or
const imageElement = document.createElement('img');
imageElement.src = URL.createObjectURL(jpegFile);
document.body.appendChild(imageElement);
Note that the heic2jpeg library requires the canvas and process modules to be available in the global context, so you may need to include these modules in your application as well.

html2canvas failing to capture Google static map

I'm creating a booking reference for the user to download after successfully making a booking. It looks like this in the browser:
However, after using html2canvas to get a screenshot of the component in React, I get this:
I suspect the issue has to do with the fact that the static map's use is protected by an API key, causing issues to render the image in an external view.
This is the callback that fires on hitting the share button, which for now just downloads the png (which is then converted to pdf using jsPDF):
const share = () => {
const input = document.getElementById("trip-summary");
if (input === null) return;
html2canvas(input!).then((canvas) => {
var image = canvas
.toDataURL("image/png")
.replace("image/png", "image/octet-stream");
const pdf = new jsPDF();
// #ts-ignore
pdf.addImage(image, "PNG", 0, 0);
pdf.save("download.pdf");
});
};
It looks like an external source of content. You should set the allowTaint option to true.
html2canvas(input, { allowTaint: true }).then(...);
Just wanted to add to the answer above.
It achieves exactly what I want, but you just need to set useCORS to true in the options as well
html2canvas(input!, { allowTaint: true, useCORS: true }).then(...)
, otherwise you'll get
Uncaught SecurityError: Failed to execute 'toDataURL' on 'HTMLCanvasElement': Tainted canvases may not be exported.

React APP makes a ton of API Calls and crashes

I am making a simple app that makes an api request to my local server and gets some data and puts it on a chart from trading view. This should be pretty simple as everything is just for practice, but when I change some of the values on my server and make the call, the app keeps making the call like 35 times before the server crashes and then the app just says
"net::ERR_CONNECTION_REFUSED"
and doesn't display the data as it should.
This is the whole code, it has two parts. One parts makes the call to get example data of name and another call to get example data that will go to the chart (the second part is the issue.)
This is the code just for the second part:
getBars: async (
symbolInfo,
resolution,
periodParams,
onHistoryCallback,
onErrorCallback
) => {
try {
if (resolution === '1D') {
resolution = 1440;
}
const response2 = await axios.get('http://localhost:8000/chart');
console.log('got bars data');
const bars = response2.data.map((el) => ({
time: new Date(el.timeInterval.minute).getTime(), // date string in api response
low: el.low,
high: el.high,
open: Number(el.open),
close: Number(el.close),
volume: el.volume,
}));
if (bars.length) {
onHistoryCallback(bars, { noData: false });
} else {
onHistoryCallback(bars, { noData: true });
}
console.log('bars done');
} catch (err) {
console.log({ err });
}
};
So what happens is that the console.log "got bars data" and "bars done" repeats many times until my localhost:8000 server crashes and then the app gives the error I showed above, because of this it doesn't display the data. I have no Idea why this may be happening,
This is what the data looks like for the one that does not works:
{"timeInterval":{"minute":"2022-03-14T23:45:00Z"},"volume":0.05,"high":3.910209183178435e-9,"low":3.910209183178435e-9,"open":"3.910209183178435e-09","close":"3.910209183178435e-09"}
This is for the one that works:
{"timeInterval":{"minute":"2022-03-17T15:00:00Z"},"volume":0.05,"high":0.00001255389794727055,"low":0.00001255389794727055,"open":"1.255389794727055e-05","close":"1.255389794727055e-05"}
I would appreciate any help, thanks!
EDIT
I just noticed, with the data set that works, console.log('got bars data') and console.log('bars done') don't occur for some reason, but the data still shows up on the chart even though the console doesn't log.

convert PDF data into octet-stream for printer autodetection

I have a printer that only accepts application/octet-stream over IPP. My program downloads binary PDF data and I need to convert that into application/octet-stream, which would (supposedly) let the printer decide what to print. However, when I send the data, it just prints binary data as text and not as formatted PDF. I'm using node with npm package 'ipp'.
I had a problem similar to that, in this link! I found a working example that, I modified a little like this to work (mixed some with the pdfkit! example, but shorted).
Here is my working version (node v16.17.0 | npm 8.15.0 | windows 11)
var ipp = require("ipp");
var concat = require("concat-stream");
var PDFDocument = require('pdfkit');
const doc = new PDFDocument();
// Pipe its output somewhere, like to a file or HTTP response
// Render some text
doc
.fontSize(25)
.text('Some text with an embedded font!', 100, 100);
// Add an image, constrain it to a given size, and center it vertically and horizontally
doc.image('./my-image.png', {
fit: [250, 300],
align: 'center',
valign: 'center'
});
doc.pipe(concat(function (data) {
//I used this url with a Brother printer, because the 631 port had some weird problem
var printer = ipp.Printer("http://<ip address>:80/ipp",{version:'2.0'});
var file = {
"operation-attributes-tag":{
"requesting-user-name": "User",
"job-name": "Print Job",
"document-format": "application/octet-stream"
},
data: data
};
printer.execute("Print-Job", file, function (err, res) {
//in case of error
console.log("Error: ",err);
console.log('res',res);
});
}));
//This last line is very important!
doc.end();
note that the version you have to check if your printer supports it
I checked that with this code: (I lost the link where I found this, so that is why there is not reference to it)
var ipp = require('ipp');
var uri = "http://<ip address>:80/ipp";
var data = ipp.serialize({
"operation":"Get-Printer-Attributes",
"operation-attributes-tag": {
"attributes-charset": "utf-8",
"attributes-natural-language": "en",
"printer-uri": uri
}
});
ipp.request(uri, data, function(err, res){
if(err){
return console.log(err);
}
console.log(JSON.stringify(res,null,2));
})

Perform Asynchronous Decorations in DraftJS?

I'm trying to perform real-time Named Entity Recognition highlighting in a WYSIWYG editor, which requires me to make a request to my back-end in between each keystroke.
After spending about a week on ProseMirror I gave up on it and decided to try DraftJS. I have searched the repository and docs and haven't found any asynchronous examples using Decorations. (There are some examples with Entities, but they seem like a bad fit for my problem.)
Here is the stripped down Codepen of what I'd like to solve.
It boils down to me wanting to do something like this:
const handleStrategy = (contentBlock, callback, contentState) => {
const text = contentBlock.getText();
let matchArr, start;
while ((matchArr = properNouns.exec(text)) !== null) {
start = matchArr.index;
setTimeout(() => {
// THROWS ERROR: Cannot read property '0' of null
callback(start, start + matchArr[0].length);
}, 200) // to simulate API request
}
};
I expected it to asynchronously call the callback once the timeout resolved but instead matchArr is empty, which just confuses me.
Any help is appreciated!
ok, one possible solution, a example, simple version (may not be 100% solid) :
write a function take editor's string, send it to server, and resolve the data get from server, you need to figure out send the whole editor string or just one word
getServerResult = data => new Promise((resolve, reject) => {
...
fetch(link, {
method: 'POST',
headers: {
...
},
// figure what to send here
body: this.state.editorState.getCurrentContent().getPlainText(),
})
.then(res => resolve(res))
.catch(reject);
});
determine when to call the getServerResult function(i.e when to send string to server and get entity data), from what I understand from your comment, when user hit spacebar key, send the word before to server, this can done by draftjs Key Bindings or react SyntheticEvent. You will need to handle case what if user hit spacebar many times continuously.
function myKeyBindingFn(e: SyntheticKeyboardEvent): string {
if (e.keyCode === 32) {
return 'send-server';
}
return getDefaultKeyBinding(e);
}
async handleKeyCommand(command: string): DraftHandleValue {
if (command === 'send-server') {
// you need to manually add a space char to the editorState
// and get result from server
...
// entity data get from server
const result = await getServerResult()
return 'handled';
}
return 'not-handled';
}
add entity data get from server to specific word using ContentState.createEntity()
async handleKeyCommand(command: string): DraftHandleValue {
if (command === 'send-server') {
// you need to manually add a space char to the editorState
// and get result from server
...
// entity data get from server
const result = await getServerResult()
const newContentState = ContentState.createEntity(
type: 'string',
mutability: ...
data: result
)
const entityKey = contentStateWithEntity.getLastCreatedEntityKey();
// you need to figure out the selectionState, selectionState mean add
// the entity data to where
const contentStateWithEntity = Modifier.applyEntity(
newContentState,
selectionState,
entityKey
);
// create a new EditorState and use this.setState()
const newEditorState = EditorState.push(
...
contentState: contentStateWithEntity
)
this.setState({
editorState: newEditorState
})
return 'handled';
}
return 'not-handled';
}
create different decorators find words with specific entity data, and return different style or whatever you need to return
...
const compositeDecorator = new CompositeDecorator([
strategy: findSubjStrategy,
component: HandleSubjSpan,
])
function findSubjStrategy(contentBlock, callback, contentState) {
// search whole editor content find words with subj entity data
// if the word's entity data === 'Subj'
// pass the start index & end index of the word to callback
...
if(...) {
...
callback(startIndex, endIndex);
}
}
// this function handle what if findSubjStrategy() find any word with subj
// entity data
const HandleSubjSpan = (props) => {
// if the word with subj entity data, it font color become red
return <span {...props} style={{ color: 'red' }}>{props.children}</span>;
};

Resources