I want to store 100000 record in my application so i can use it in offline mod first I try to use Asyncstorage
try{
await AsyncStorage.setItem(#Devices, JSON.stringify(data));
}
but I have problem when I try to get the saved data
let Devices = await AsyncStorage.getItem('#Devices');
let newList = await JSON.parse( Devices );
AsyncStorage throw error :
AsyncStorage Couldn't read row 0, col 0 from CursorWindow
then I try to use react-native-sqlite-storage I face problem where
I have a problem that the data must be added one by one (for loop to insert 100000 record) and this led to the application being stuck
// Didmount
useEffect(() => {
createTable();
});
const createTable = () => {
db.transaction(tx => {
tx.executeSql(
'CREATE TABLE IF NOT EXISTS' +
'USERS' +
'(ID INTEGER PRIMARY KEY AUTOINCREMENT,item_code TEXT,item_desc TEXT,ser_id TEXT,inv_no TEXT,status TEXT,location TEXT,)',
);
});
};
const insertIntoTable = () => {
deviceList.map(item=>{
db.transaction(tx => {
tx.executeSql(
'INSERT INTO USERS (item_code,item_desc,ser_id,inv_no,status,location) VALUES (?,?,?,?,?,?,)',
[item.item_code, item.item_desc, item.ser_id, iitem.nv_no, item.status, item.location],
);
});
})
};
you have to call transaction at first ,and define loop in , it's something like this :
try {
db.transaction(
tx => {
deviceList.forEach(async item => {
await tx.executeSql(
'INSERT INTO USERS (item_code,item_desc,ser_id,inv_no,status,location) VALUES (?,?,?,?,?,?,)',
[item.item_code, item.item_desc, item.ser_id, iitem.nv_no, item.status, item.location],
);
});
},
null,
null,
);
} catch (error) {
result = false;
}
maybe it's Helpful :)
I solved my problem by dividing the data into 10,000 for each one, so the 100,000 divided into 10 AsyncStorage
let loop = (await res.items.length) / 10000;
let loop2 = (await loop) % 1 === 0 ? loop : parseInt(loop) + 1;
let first = -10000;
let second = 0;
let last = res.items.length;
await AsyncStorage.setItem('loop', loop2.toString());
for (var i = 0; i < loop2; i++) {
first = first + 10000;
second = i + 1 == loop2 ? last : second + 10000;
let data = await res.items.slice(first, second);
try {
let Device = await ('#Device' + i + 1);
await AsyncStorage.setItem(Device, JSON.stringify(data));
if (i + 1 == loop2) {
setIsLoading(false);
}
Related
I want to select 100 audio file at a time but want to hit only 3 api call at a time. Once these 3 files uploaded (pass or fail) then only other 3 api request will be sent.
Basically I am providing a input field of file type:
<input type="file" multiple name="file" className="myform"
onChange={handleFileChange}
accept="audio/wav"
/>
and I am storing it as array into a state.
Below this I am providing an UPLOAD button.
When user hit on upload, I want to send 3 POST request using axios. once all 3 done either fail or pass then only next 3 should go.
You can do this by iterating the FileList collection in groups of 3 and sending the requests in parallel using Promise.allSettled().
Simply because I cannot recommend Axios, here's a version using the Fetch API
const BATCH_SIZE = 3;
const [fileList, setFileList] = useState([]);
const [uploading, setUploading] = useState(false);
const handleFileChange = (e) => {
setFileList(Array.from(e.target.files)); // just a guess
};
const handleUploadClick = async (e) => {
e.preventDefault();
setUploading(true);
const files = [...fileList]; // avoid mutation during long uploading process
for (let i = 0; i < files.length; i += BATCH_SIZE) {
const result = await Promise.allSettled(
files.slice(i, i + BATCH_SIZE).map(async (file) => {
const body = new FormData();
body.append("file", file);
const res = await fetch(UPLOAD_URL, { method: "POST", body });
return res.ok ? res : Promise.reject(res);
})
);
const passed = result.filter(({ status }) => status === "fulfilled");
console.log(
`Batch ${i + 1}: ${
passed.length
} of ${BATCH_SIZE} requests uploaded successfully`
);
}
setUploading(false);
};
Promise.allSettled() will let you continue after each set of 3 are uploaded, whether they pass or fail.
This method makes 3 separate requests with 1 file each.
With Axios, it would look like this (just replacing the for loop)
for (let i = 0; i < files.length; i += BATCH_SIZE) {
const result = await Promise.allSettled(
files
.slice(i, i + BATCH_SIZE)
.map((file) => axios.postForm(UPLOAD_URL, { file }))
);
const passed = result.filter(({ status }) => status === "fulfilled");
console.log(
`Batch ${i + 1}: ${
passed.length
} of ${BATCH_SIZE} requests uploaded successfully`
);
}
Axios' postForm() method is available from v1.0.0. See https://github.com/axios/axios#files-posting
If you want to send 3 files in a single request, it would look like this for Fetch
for (let i = 0; i < files.length; i += BATCH_SIZE) {
const body = new FormData();
files.slice(i, i + BATCH_SIZE).forEach((file) => {
body.append("file", file); // use "file[]" for the first arg if required
});
try {
const res = await fetch(UPLOAD_URL, { method: "POST", body });
if (!res.ok) {
throw new Error(`${res.status} ${res.statusText}`);
}
console.log(`Batch ${i + 1} passed`);
} catch (err) {
console.warn(`Batch ${i + 1} failed`, err);
}
}
and this for Axios
for (let i = 0; i < files.length; i += BATCH_SIZE) {
try {
await axios.postForm(
{
file: files.slice(i, i + BATCH_SIZE),
},
{
formSerializer: {
indexes: null, // set to false if you need "[]" added
},
}
);
console.log(`Batch ${i + 1} passed`);
} catch (err) {
console.warn(`Batch ${i + 1} failed`, err.response?.data);
}
}
You can use a combination of JavaScript's for loop and Promise.all functions to achieve this. First, you will need to divide your files array into chunks of 3. You can do this using a for loop and the slice method. Next, you can use Promise.all to send all the requests in parallel, and only move on to the next set of requests once all the promises in the current set have been resolved. Here's some sample code that demonstrates this approach:
const chunkSize = 3;
for (let i = 0; i < files.length; i += chunkSize) {
const fileChunk = files.slice(i, i + chunkSize);
const promises = fileChunk.map(file => {
return axios.post('/api/upload', { file });
});
await Promise.all(promises);
}
This will send 3 post request at a time and will wait until all the request are completed before sending another 3 api request.
You can also use useState hook with useEffect to set the state of files that are uploaded and use a variable to keep track of number of files uploaded.
const [uploadedFiles, setUploadedFiles] = useState([]);
const [uploadCount, setUploadCount] = useState(0);
useEffect(() => {
if (uploadCount === files.length) {
// all files have been uploaded
return;
}
const chunkSize = 3;
const fileChunk = files.slice(uploadCount, uploadCount + chunkSize);
const promises = fileChunk.map(file => {
return axios.post('/api/upload', { file });
});
Promise.all(promises).then(responses => {
setUploadedFiles([...uploadedFiles, ...responses]);
setUploadCount(uploadCount + chunkSize);
});
}, [uploadCount]);
This code will work for you.
my function getContents results in the error:"Query was already executed: Content.find({})"
why am i getting this error? how can i fix it? i recently changed this function and this was the byproduct.
exports.getContents = asyncErrors (async (req,res,next) =>{
const contentsCount = await Content.countDocuments(); //162
const resPerPage = 9;
const apiFeatures = new APIFeatures(Content.find(),req.query).search().filter()
console.log(" api " + apiFeatures)
let contents = await apiFeatures.query;
console.log(' contentslist ' + contents)
let filteredContentCount = contents.length;
console.log(' filtered ' + filteredContentCount)
apiFeatures.pagination(resPerPage)
contents = await apiFeatures.query;
/* */
// let cont = await Content.find()
// let cont2 = await Promise.all (cont.map(async (item) =>{
// item.set('content-Count',await Content.find({categories:item._id}).countDocuments(),{strict:false});
// return item
// }))
// console.log(' cont ' + cont2)
/* */
/* check for category request
assign total amount of objects within a category to a variable/constant
if(category.req){
let contentCount = category.counter
}
*/
setTimeout(() => {
res.status(200).json({
success:true,
contentsCount,
resPerPage,
filteredContentCount,
contents
})
}, 2000);
})
I have found that it could be a call back issue with async and await. but i am lost on whether that applies here
so I have this working fetch I wanted to know if I could improve it more so it will be faster, so the api only allows a limit of 100 items to return per call, and has a skipQ to skip a number so we could skip the previous responses and not get duplication - so what I did was do a while loop until the response from one of the fields (dataCount) returns 0. Thoughts on improving and other alternatives?
const fetchUsers = async () => {
let d = 1
let skip = 0
while (d !== 0) {
const response = await axios.post(`${URI}, {limit: '100', skipQ: skip}`)
skip += 100
if(response.data.dataCount === 0) {
d = response.data.dataCount
}
setUsers(prev => [...prev, response.data].flat())
} else {
navigate('/')
}
}
fetchUsers()
this function is inside a useEffect hhok
I'm trying to iterate through some products and then assignate the summatory of the elements to an object.
This is my code, but it seems like the assignation is running first and then everything inside the forEach.
How could I wait until the forEach has finished in order to apply the correct value to my variables?
const megaCheck = async (prods) => {
let puntosGastados = 0
let stickersGastados = 0
const productos = prods[0].items
let host = '' if (
window.location.host.split('.')[0].indexOf('wongfoodqawlschv6io') > -1 ) {
host = 'wongfoodqawlschv6io' } else if (window.location.host.split('.')[0].indexOf('metroqaio') > -1) {
host = 'metroqaio' }
productos.forEach(async (producto, index) => {
try {
const { data } = await axios.get(`/api/catalog_system/pub/products/search?fq=productId:${producto.id}`)
if(data[0]["mega-promo"].length){
let megaInfo = JSON.parse(data[0]["mega-promo"])
puntosGastados += (megaInfo.puntos * producto.quantity)
stickersGastados += (megaInfo.stickers * producto.quantity)
console.log("Puntos gastados --" + puntosGastados)
console.log("Stickers gastados --" + stickersGastados)
}
} catch (error) {
console.error(error)
} })
console.log(puntosGastados)
console.log(stickersGastados)
setMegaUserInfo({
points: puntosGastados,
stickers: stickersGastados })
console.log(megaUserInfo) }
And my output is: OUTPUT
You can wait for every iteration to be completed using Promise.all
Promise.all() documentation
await Promise.all(
// Replacing forEach with map because we need to get the promise of each iteration
productos.map(async (producto, index) => {
try {
const { data } = await axios.get(
`/api/catalog_system/pub/products/search?fq=productId:${producto.id}`
);
if (data[0]["mega-promo"].length) {
let megaInfo = JSON.parse(data[0]["mega-promo"]);
puntosGastados += megaInfo.puntos * producto.quantity;
stickersGastados += megaInfo.stickers * producto.quantity;
console.log("Puntos gastados --" + puntosGastados);
console.log("Stickers gastados --" + stickersGastados);
}
} catch (error) {
console.error(error);
}
})
);
console.log(puntosGastados);
console.log(stickersGastados);
I believe that you would be able to achieve what you want by using the useRef hook
puntosGastados = useRef(0)
stickersGastados = useRef(0)
...
puntosGastados.current += (megaInfo.puntos * producto.quantity)
stickersGastados.current += (megaInfo.stickers * producto.quantity)
https://reactjs.org/docs/hooks-reference.html#useref
I'm trying to bring rows of data back from an SQLite database and then iterate through them so I can manipulate the data of each row to present them differently (ie, convert a date into a customised format).
Bringing the data back is fine (from the ReadEntries function), which I do in a useEffect so that it only runs once on the screen load, but the duplicating of the array and then updating the rows doesn't seem to work. I think it might have something to do with the fact setting the value of a useState array isn't quick enough to update for my duplicate array to then take a full snapshot of it.
It sometimes works when I save multiple times in VS Code, presumably because the state is already stored for subsequent screen refreshes.
useEffect(() => {
var results: SQLite.ResultSetRowList;
const response = async (): Promise<any> => {
await ReadEntries(projectID).then((value) => {
results = value as SQLite.ResultSetRowList;
setEntries(results.raw); //this works
let newEntries = [...entries];
for (let i = 0; i < newEntries.length; i++) {
let newStartDate = new Date(newEntries[i].startDate);
newEntries[i].dateOfEntry = newEntries[i].dateOfEntry;
newEntries[i].startDate =
newStartDate.getDate().toString() +
"/" +
newStartDate.getMonth().toString() +
"/" +
newStartDate.getFullYear().toString();
}
setEntries(newEntries);
});
};
response().catch((error) => {
"ERROR: " + error;
});
}, []);
Thanks
State updates with hooks are not only async but are also bound by closure. You should not reply on the updated state to trigger another change in the same function call
Check this post for more details on this: useState set method not reflecting change immediately
You can make use of the the fetched data to perform the operation
useEffect(() => {
var results: SQLite.ResultSetRowList;
const response = async (): Promise<any> => {
await ReadEntries(projectID).then((value) => {
results = value as SQLite.ResultSetRowList;
setEntries(results.raw); // This shouldn't be required.
let newEntries = [...results.raw]; // use results instead of entries
for (let i = 0; i < newEntries.length; i++) {
let newStartDate = new Date(newEntries[i].startDate);
newEntries[i].dateOfEntry = newEntries[i].dateOfEntry;
newEntries[i].startDate =
newStartDate.getDate().toString() +
"/" +
newStartDate.getMonth().toString() +
"/" +
newStartDate.getFullYear().toString();
}
setEntries(newEntries);
});
};
response().catch((error) => {
"ERROR: " + error;
});
}, []);
I achieved this by calling the .raw() function of the returned results, and looping through that array with forEach. I then created a new object (newarrayItem) and assigned the fields to the .raw() array values, then pushed that object into a blank array (newarray) that I assigned the type TentryItemString. Then passed this to the setEntries useState array.
useEffect(() => {
var results: SQLite.ResultSetRowList;
let newarray: TentryItemString[] = [];
const response = async (): Promise<any> => {
await ReadEntries(projectID).then((value) => {
results = value as SQLite.ResultSetRowList;
results.raw().forEach((element, index) => {
let newdate = "";
let newStartDate = new Date(element.startDate);
newdate =
newStartDate.getDate().toString() +
"/" +
(newStartDate.getMonth() + 1).toString() +
"/" +
newStartDate.getFullYear().toString();
var newarrayItem = {
dateOfEntry: "",
startDate: "",
};
newarrayItem.startDate = newdate;
newarrayItem.dateOfEntry = element.dateOfEntry;
newarray.push(newarrayItem);
});
});
newarray.reverse();
setEntries(newarray);
};
response().catch((error) => {
"ERROR: " + error;
});
}, []);
A couple of issues I found in doing this. One, if I initiated the newarrayItem by equalling it to an already created object (ie. var newarray = newarrayItem), the array.push() method seemed to overwrite all rows with the same information. That's why I've instantiated it there and then. The other issue was struggling with assigning the type to the useState as an array, but I finally achieved this with the following syntax:
const [entries, setEntries] = useState();
And all TentryItemString is, is:
type TentryItemString = {
dateOfEntry: string;
startDate: string;
};
Hope someone finds that helpful.