Because of the bug https://github.com/codenameone/CodenameOne/issues/3043, I don't know how to show upload percentage when using a MultipartRequest. Do you have any suggestion, like an alternative way to show the percentage? Thank you
I solved this issue, workarounding it. After spending some days trying client-only solutions, finally I ended up in a solution that involves client code (Codename One) and server code (Spring Boot).
Basically, on the client I split the file to be upload in small pieces of 100kb and I upload them one by one, so I can calculate exactly the uploaded percentage. On the server, I put a controller to receive the small files and another controller to merge them. I know that my code is specific for my use case (sending images and videos to Cloudinary), however I copy some relevant parts that could inspire other people that have a similar problem with Codename One.
Screenshot ("Caricamento" means "Uploading" and "Annulla" means "Cancel"):
Client code
Server class
/**
* SYNC - Upload a MultipartFile as partial file
*
* #param data
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #return true if success, false otherwise
*/
public static boolean uploadPartialFile(byte[] data, int partNumber, String uniqueId) {
String api = "/cloud/partialUpload";
MultipartRequest request = new MultipartRequest();
request.setUrl(Server.getServerURL() + api);
request.addData("file", data, "application/octet-stream");
request.addRequestHeader("authToken", DB.userDB.authToken.get());
request.addRequestHeader("email", DB.userDB.email.get());
request.addRequestHeader("partNumber", partNumber + "");
request.addRequestHeader("uniqueId", uniqueId);
NetworkManager.getInstance().addToQueueAndWait(request);
try {
String response = Util.readToString(new ByteArrayInputStream(request.getResponseData()), "UTF-8");
if ("OK".equals(response)) {
return true;
}
} catch (IOException ex) {
Log.p("Server.uploadPartialFile ERROR -> Util.readToString failed");
Log.e(ex);
SendLog.sendLogAsync();
}
return false;
}
/**
* ASYNC - Merges the previously upload partial files
*
* #param uniqueId containing the totalBytes before the first "-"
* #param callback to do something with the publicId of the uploaded file
*/
public static void mergeUpload(String uniqueId, OnComplete<Response<String>> callback) {
String api = "/cloud/mergeUpload";
Map<String, String> headers = Server.getUserHeaders();
headers.put("uniqueId", uniqueId);
Server.asyncGET(api, headers, callback);
}
public static void uploadFile(String filePath, OnComplete<String> callback) {
String api = "/cloud/upload";
// to show the progress, we send a piece of the file at a time
String url = Server.getServerURL() + api;
Map<String, String> headers = new HashMap<>();
headers.put("authToken", DB.userDB.authToken.get());
headers.put("email", DB.userDB.email.get());
DialogUtilities.genericUploadProgress(url, filePath, headers, callback);
}
}
DialogUtilities class
public static void genericUploadProgress(String url, String filePath, Map<String, String> headers, OnComplete<String> callback) {
Command[] cmds = {Command.create("Cancel", null, ev -> {
((Dialog) Display.getInstance().getCurrent()).dispose();
uploadThread.kill();
})};
Container bodyCmp = new Container(new BorderLayout());
Label infoText = new Label("DialogUtilities-Upload-Starting");
bodyCmp.add(BorderLayout.CENTER, infoText);
// Dialog blocks the current thread (that is the EDT), so the following code needs to be run in another thread
uploadThread.run(() -> {
// waits some time to give the Dialog the time to be open
// it's not necessary, but useful to use the SelectorUtilities below in the case that the uploaded file is very small
Util.sleep(500);
try {
long size = FileSystemStorage.getInstance().getLength(filePath);
String uniqueId = size + "-" + DB.userDB.email + "_" + System.currentTimeMillis();
// splits the file in blocks of 100kb
InputStream inputStream = FileSystemStorage.getInstance().openInputStream(filePath);
byte[] buffer = new byte[100 * 1024];
int readByte = inputStream.read(buffer);
int totalReadByte = 0;
int partNumber = 0;
while (readByte != -1) {
boolean result = Server.uploadPartialFile(Arrays.copyOfRange(buffer, 0, readByte), partNumber, uniqueId);
if (!result) {
CN.callSerially(() -> {
DialogUtilities.genericServerError();
});
break;
}
partNumber++;
totalReadByte += readByte;
int percentage = (int) (totalReadByte * 100 / size);
CN.callSerially(() -> {
infoText.setText(percentage + "%");
});
readByte = inputStream.read(buffer);
}
CN.callSerially(() -> {
if (CN.getCurrentForm() instanceof Dialog) {
// upload finished, before merging the files on the server we disable the "Cancel" button
Button cancelBtn = SelectorUtilities.$(Button.class, CN.getCurrentForm()).iterator().next();
cancelBtn.setEnabled(false);
cancelBtn.setText("DialogUtilities-Wait");
cancelBtn.repaint();
}
});
Server.mergeUpload(uniqueId, new OnComplete<Response<String>>() {
#Override
public void completed(Response<String> response) {
String fileId = response.getResponseData();
CN.callSerially(() -> {
if (Display.getInstance().getCurrent() instanceof Dialog) {
((Dialog) Display.getInstance().getCurrent()).dispose();
}
});
callback.completed(fileId);
}
});
} catch (IOException ex) {
Log.p("DialogUtilities.genericUploadProgress ERROR", Log.ERROR);
CN.callSerially(() -> {
DialogUtilities.genericDialogError("DialogUtilities-UploadError-Title", "DialogUtilities-UploadError-Text");
});
Log.e(ex);
SendLog.sendLogAsync();
}
});
showDialog("Server-Uploading", null, cmds[0], cmds, DialogUtilities.TYPE_UPLOAD, null, 0l, CommonTransitions.createDialogPulsate().copy(false), null, null, bodyCmp);
Server code
CloudinaryController class
/**
* Upload a MultipartFile as partial file.
*
* #param authToken
* #param email
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #param file
* #return "OK" if success
*/
#PostMapping("/partialUpload")
public #ResponseBody
String partialUpload(#RequestHeader(value = "authToken") String authToken, #RequestHeader(value = "email") String email, #RequestHeader(value = "partNumber") String partNumber, #RequestHeader(value = "uniqueId") String uniqueId, #RequestParam("file") MultipartFile file) throws IOException {
return cloudinaryService.partialUpload(authToken, email, partNumber, uniqueId, file);
}
/**
* Merges the files previuosly uploaded by "/partialUpload", upload that
* file to Cloudinary and returns the id assigned by Cloudinary
*
* #param authToken
* #param email
* #param uniqueId containing the totalBytes before the first "-"
* #return the id assigned by Cloudinary
*/
#GetMapping("/mergeUpload")
public #ResponseBody
String mergeUpload(#RequestHeader(value = "authToken") String authToken, #RequestHeader(value = "email") String email, #RequestHeader(value = "uniqueId") String uniqueId) throws IOException {
return cloudinaryService.mergeUpload(authToken, email, uniqueId);
}
CloudinaryService class
/**
* Upload a MultipartFile as partial file.
*
* #param authToken
* #param email
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #param file
* #return "OK" if success
*/
public String partialUpload(String authToken, String email, String partNumber, String uniqueId, MultipartFile file) throws IOException {
User user = userService.getUser(authToken, email);
if (user != null) {
String output = AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber;
Path destination = Paths.get(output);
Files.copy(file.getInputStream(), destination, StandardCopyOption.REPLACE_EXISTING);
return "OK";
} else {
logger.error("Error: a not authenticated user tried to upload a file (email: " + email + ", authToken: " + authToken + ")");
return null;
}
}
/**
* Merges the files previuosly uploaded by "/partialUpload", upload that
* file to Cloudinary and returns the id assigned by Cloudinary
*
* #param authToken
* #param email
* #param uniqueId containing the totalBytes before the first "-"
* #return the id assigned by Cloudinary
*/
public String mergeUpload(String authToken, String email, String uniqueId) throws IOException {
User user = userService.getUser(authToken, email);
if (user != null) {
long totalBytes = Long.valueOf(uniqueId.split("-", 2)[0]);
List<File> files = new ArrayList<>();
int partNumber = 0;
File testFile = new File(AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber);
while (testFile.exists()) {
files.add(testFile);
partNumber++;
testFile = new File(AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber);
}
// the list of files is ready, we can now merge them
File merged = new File(AppApplication.uploadTempDir + "/" + uniqueId);
IOCopier.joinFiles(merged, files);
// uploads the file to Cloudinary
Map uploadResult = cloudinary.uploader().upload(merged, ObjectUtils.emptyMap());
String publicId = uploadResult.get("public_id").toString();
// removes the files
for (File file : files) {
file.delete();
}
merged.delete();
return publicId;
} else {
logger.error("Error: a not authenticated user tried to upload a file (email: " + email + ", authToken: " + authToken + ")");
return null;
}
}
IOCopier class
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;
import org.apache.commons.io.IOUtils;
/**
* Useful to merge files. See: https://stackoverflow.com/a/14673198
*/
public class IOCopier {
public static void joinFiles(File destination, List<File> sources)
throws IOException {
OutputStream output = null;
try {
output = createAppendableStream(destination);
for (File source : sources) {
appendFile(output, source);
}
} finally {
IOUtils.closeQuietly(output);
}
}
private static BufferedOutputStream createAppendableStream(File destination)
throws FileNotFoundException {
return new BufferedOutputStream(new FileOutputStream(destination, true));
}
private static void appendFile(OutputStream output, File source)
throws IOException {
InputStream input = null;
try {
input = new BufferedInputStream(new FileInputStream(source));
IOUtils.copy(input, output);
} finally {
IOUtils.closeQuietly(input);
}
}
}
Not at this time as there's no evaluation for where the issue lies. I think the progress listener tracks the output stream writing to the upload code not the actual connection time which is normally hard to track in Java anyway.
E.g. in Java SE you would open a URL and then write to the output stream of a POST connection. Then the writing would actually occur when you try to get the input stream response. But at this point I would have no indication about the state of the upload as it's completely abstracted and happening under the hood.
So I'm not sure if this is even technically feasible.
i have a rest api with spring boot that works fine it uploads an excel file in temporary file then it adds it to the database
However i am a beginner in angular Js so i am not able to do that even after a long search on the internet i will appreciate any help
here is my code for the rest API
#RestController
public class UploadController {
#Autowired
PosteDao posteDao ;
//Save the uploaded file to this folder
private static String UPLOADED_FOLDER = "C://Temp//";
#PostMapping("/doUploadFile") // //new annotation since 4.3
public String singleFileUpload(#RequestParam("file") MultipartFile file,
RedirectAttributes redirectAttributes) {
if (file.isEmpty()) {
redirectAttributes.addFlashAttribute("message", "Please select a file to upload");
return "redirect:uploadStatus";
}
try {
// Get the file and save it somewhere
byte[] bytes = file.getBytes();
Path path = Paths.get(UPLOADED_FOLDER + file.getOriginalFilename());
Files.write(path, bytes);
redirectAttributes.addFlashAttribute("message",
"You successfully uploaded '" + file.getOriginalFilename() + "'");
processExcel();
} catch (IOException e) {
e.printStackTrace();
}
return "greaat";
}
public String processExcel() throws IOException {
try {
InputStream ExcelFileToRead = new FileInputStream("C:\\\\list.xlsx");
//définir l'objet workbook
XSSFWorkbook wb = new XSSFWorkbook(ExcelFileToRead);
//Itérer sur le nombre de sheets
for(int i=0;i<wb.getNumberOfSheets();i++)
{
XSSFSheet sheet = wb.getSheetAt(i);
//Itérer sur les lignes
Row row;
for(int k=1;k<sheet.getLastRowNum()+1;k++)
{
row = (Row) sheet.getRow(k); //sheet number
int idPoste;
if( row.getCell(0)==null) { idPoste = 0; }
else idPoste= (int) row.getCell(0).getNumericCellValue();
String libelle;
if( row.getCell(1)==null) { libelle = "null";} //suppose excel cell is empty then its set to 0 the variable
else libelle = row.getCell(1).toString(); //else copies cell data to name variable
System.out.println(libelle);
System.out.println(idPoste);
}
}} catch (Exception e) {
System.err.println("Erreur");
// TODO Auto-generated catch block
e.printStackTrace();
}
return "data extracted successfully";
}
}
I am trying to connect my app to database on localhost server.I can connect to it ut the problem is how to pass the parameter from app to php script.for eg i want all names having age less than 10 so i will pass the parameter to php.below is my code for connecting to database.please provide good reference
/* */
enter code here
public class TestExternalDatabaseActivity extends Activity {
/** Called when the activity is first created. */
TextView resultView;
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
StrictMode.enableDefaults(); //STRICT MODE ENABLED
resultView = (TextView) findViewById(R.id.result);
getData();
}
public void getData(){
String result = "";
InputStream isr = null;
try{
HttpClient httpclient = new DefaultHttpClient();
HttpPost httppost = new HttpPost("http://localhost/tahseen0amin/php/getAllCustomers.php"); //YOUR PHP SCRIPT ADDRESS
HttpResponse response = httpclient.execute(httppost);
HttpEntity entity = response.getEntity();
isr = entity.getContent();
}
catch(Exception e){
Log.e("log_tag", "Error in http connection "+e.toString());
resultView.setText("Couldnt connect to database");
}
//convert response to string
try{
BufferedReader reader = new BufferedReader(new InputStreamReader(isr,"iso-8859-1"),8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
isr.close();
result=sb.toString();
}
catch(Exception e){
Log.e("log_tag", "Error converting result "+e.toString());
}
//parse json data
try {
String s = "";
JSONArray jArray = new JSONArray(result);
for(int i=0; i<jArray.length();i++){
JSONObject json = jArray.getJSONObject(i);
s = s +
"Name : "+json.getString("FirstName")+" "+json.getString("LastName")+"\n"+
"Age : "+json.getInt("Age")+"\n"+
"Mobile Using : "+json.getString("Mobile")+"\n\n";
}
resultView.setText(s);
} catch (Exception e) {
// TODO: handle exception
Log.e("log_tag", "Error Parsing Data "+e.toString());
}
}
}
I want to create a method that will load a txt file and then change it but thats another method.
private void openFile() {
fileChooser.getSelectedFile();
JFileChooser openFile = new JFileChooser();
openFile.showOpenDialog(frame);
}
What must go next in order to get data from the file after choosing it to manipulate its data?
The JFileChooser documentation has an example on how to continue your code, and get the name of the file chosen, which can then be turned into a File object. You should be able to modify that example to meet your needs:
JFileChooser chooser = new JFileChooser();
FileNameExtensionFilter filter = new FileNameExtensionFilter(
"JPG & GIF Images", "jpg", "gif");
chooser.setFileFilter(filter);
int returnVal = chooser.showOpenDialog(parent);
if(returnVal == JFileChooser.APPROVE_OPTION) {
System.out.println("You chose to open this file: " +
chooser.getSelectedFile().getName());
}
Here's an example that might help you. I would want to read up on and try some simple examples on different buffers that will read and write. In fact, i have worked with these a lot in the last few months and I still have to go and look.
public class ReadWriteTextFile {
static public String getContents(File aFile) {
StringBuilder contents = new StringBuilder();
try {
BufferedReader input = new BufferedReader(new FileReader(aFile));
try {
String line = null; //not declared within while loop
while (( line = input.readLine()) != null){
contents.append(line);
contents.append(System.getProperty("line.separator"));
}
}
finally {
input.close();
}
}
catch (IOException ex){
ex.printStackTrace();
}
return contents.toString();
}
static public void setContents(File aFile,
String aContents)
throws FileNotFoundException,
IOException {
if (aFile == null) {
throw new IllegalArgumentException("File should not be null.");
}
if (!aFile.exists()) {
throw new FileNotFoundException ("File does not exist: " + aFile);
}
if (!aFile.isFile()) {
throw new IllegalArgumentException("Should not be a directory: " + aFile);
}
if (!aFile.canWrite()) {
throw new IllegalArgumentException("File cannot be written: " + aFile);
}
Writer output = new BufferedWriter(new FileWriter(aFile));
try {
output.write( aContents );
}
finally {
output.close();
}
}
public static void main (String... aArguments) throws IOException {
File testFile = new File("C:\\Temp\\test.txt");//this file might have to exist (I am not
//certain but you can trap the error with a
//TRY-CATCH Block.
System.out.println("Original file contents: " + getContents(testFile));
setContents(testFile, "The content of this file has been overwritten...");
System.out.println("New file contents: " + getContents(testFile));
}
}
I'm trying to create a servlet that is able to unzip a folder which contains 3 csv files and then print out the data of each csv file.
I have been trying to use ZipInputStream but it does not provide me the capability of reading/printing content of each csv.
As i'm building this web app on GAE, I'm unable to use FileOutputStream.
Are there ways to use ZipInputStream to unzip and read individual csv without the need to create a csv on GAE?
public class AdminBootStrap extends HttpServlet {
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws IOException {
resp.setContentType("text/plain");
PrintWriter out = resp.getWriter();
try {
ServletFileUpload upload = new ServletFileUpload();
resp.setContentType("text/plain");
FileItemIterator iterator = upload.getItemIterator(req);
while (iterator.hasNext()) {
FileItemStream item = iterator.next();
InputStream in = item.openStream();
if (item.isFormField()) {
out.println("Got a form field: " + item.getFieldName());
} else {
out.println("Got an uploaded file: " + item.getFieldName() +
", name = " + item.getName());
ZipInputStream zis = new ZipInputStream(new BufferedInputStream(in));
ZipEntry entry;
// Read each entry from the ZipInputStream until no
// more entry found indicated by a null return value
// of the getNextEntry() method.
//
while ((entry = zis.getNextEntry()) != null) {
out.println("Unzipping: " + entry.getName());
//until this point, i'm only available to print each csv name.
//What I want to do is to print out the data inside each csv file.
}
}
}
} catch (Exception ex) {
ex.printStackTrace();
// throw new ServletException(ex);
}
}
}
ZipInputStream is an InputStream, so you can read from it as normal:
while ((entry = zis.getNextEntry()) {
byte[] buf = new byte[1024];
int len;
while ((len = zis.read(buf)) > 0) {
// here do something with data in buf
}