Alternative way to show upload percentage - codenameone

Question

Because of the bug https://github.com/codenameone/CodenameOne/issues/3043, I don't know how to show upload percentage when using a MultipartRequest. Do you have any suggestion, like an alternative way to show the percentage? Thank you

Answer 1

I solved this issue, workarounding it. After spending some days trying client-only solutions, finally I ended up in a solution that involves client code (Codename One) and server code (Spring Boot).
Basically, on the client I split the file to be upload in small pieces of 100kb and I upload them one by one, so I can calculate exactly the uploaded percentage. On the server, I put a controller to receive the small files and another controller to merge them. I know that my code is specific for my use case (sending images and videos to Cloudinary), however I copy some relevant parts that could inspire other people that have a similar problem with Codename One.
Screenshot ("Caricamento" means "Uploading" and "Annulla" means "Cancel"):
Client code
Server class
/**
* SYNC - Upload a MultipartFile as partial file
*
* #param data
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #return true if success, false otherwise
*/
public static boolean uploadPartialFile(byte[] data, int partNumber, String uniqueId) {
String api = "/cloud/partialUpload";
MultipartRequest request = new MultipartRequest();
request.setUrl(Server.getServerURL() + api);
request.addData("file", data, "application/octet-stream");
request.addRequestHeader("authToken", DB.userDB.authToken.get());
request.addRequestHeader("email", DB.userDB.email.get());
request.addRequestHeader("partNumber", partNumber + "");
request.addRequestHeader("uniqueId", uniqueId);
NetworkManager.getInstance().addToQueueAndWait(request);
try {
String response = Util.readToString(new ByteArrayInputStream(request.getResponseData()), "UTF-8");
if ("OK".equals(response)) {
return true;
}
} catch (IOException ex) {
Log.p("Server.uploadPartialFile ERROR -> Util.readToString failed");
Log.e(ex);
SendLog.sendLogAsync();
}
return false;
}
/**
* ASYNC - Merges the previously upload partial files
*
* #param uniqueId containing the totalBytes before the first "-"
* #param callback to do something with the publicId of the uploaded file
*/
public static void mergeUpload(String uniqueId, OnComplete<Response<String>> callback) {
String api = "/cloud/mergeUpload";
Map<String, String> headers = Server.getUserHeaders();
headers.put("uniqueId", uniqueId);
Server.asyncGET(api, headers, callback);
}
public static void uploadFile(String filePath, OnComplete<String> callback) {
String api = "/cloud/upload";
// to show the progress, we send a piece of the file at a time
String url = Server.getServerURL() + api;
Map<String, String> headers = new HashMap<>();
headers.put("authToken", DB.userDB.authToken.get());
headers.put("email", DB.userDB.email.get());
DialogUtilities.genericUploadProgress(url, filePath, headers, callback);
}
}
DialogUtilities class
public static void genericUploadProgress(String url, String filePath, Map<String, String> headers, OnComplete<String> callback) {
Command[] cmds = {Command.create("Cancel", null, ev -> {
((Dialog) Display.getInstance().getCurrent()).dispose();
uploadThread.kill();
})};
Container bodyCmp = new Container(new BorderLayout());
Label infoText = new Label("DialogUtilities-Upload-Starting");
bodyCmp.add(BorderLayout.CENTER, infoText);
// Dialog blocks the current thread (that is the EDT), so the following code needs to be run in another thread
uploadThread.run(() -> {
// waits some time to give the Dialog the time to be open
// it's not necessary, but useful to use the SelectorUtilities below in the case that the uploaded file is very small
Util.sleep(500);
try {
long size = FileSystemStorage.getInstance().getLength(filePath);
String uniqueId = size + "-" + DB.userDB.email + "_" + System.currentTimeMillis();
// splits the file in blocks of 100kb
InputStream inputStream = FileSystemStorage.getInstance().openInputStream(filePath);
byte[] buffer = new byte[100 * 1024];
int readByte = inputStream.read(buffer);
int totalReadByte = 0;
int partNumber = 0;
while (readByte != -1) {
boolean result = Server.uploadPartialFile(Arrays.copyOfRange(buffer, 0, readByte), partNumber, uniqueId);
if (!result) {
CN.callSerially(() -> {
DialogUtilities.genericServerError();
});
break;
}
partNumber++;
totalReadByte += readByte;
int percentage = (int) (totalReadByte * 100 / size);
CN.callSerially(() -> {
infoText.setText(percentage + "%");
});
readByte = inputStream.read(buffer);
}
CN.callSerially(() -> {
if (CN.getCurrentForm() instanceof Dialog) {
// upload finished, before merging the files on the server we disable the "Cancel" button
Button cancelBtn = SelectorUtilities.$(Button.class, CN.getCurrentForm()).iterator().next();
cancelBtn.setEnabled(false);
cancelBtn.setText("DialogUtilities-Wait");
cancelBtn.repaint();
}
});
Server.mergeUpload(uniqueId, new OnComplete<Response<String>>() {
#Override
public void completed(Response<String> response) {
String fileId = response.getResponseData();
CN.callSerially(() -> {
if (Display.getInstance().getCurrent() instanceof Dialog) {
((Dialog) Display.getInstance().getCurrent()).dispose();
}
});
callback.completed(fileId);
}
});
} catch (IOException ex) {
Log.p("DialogUtilities.genericUploadProgress ERROR", Log.ERROR);
CN.callSerially(() -> {
DialogUtilities.genericDialogError("DialogUtilities-UploadError-Title", "DialogUtilities-UploadError-Text");
});
Log.e(ex);
SendLog.sendLogAsync();
}
});
showDialog("Server-Uploading", null, cmds[0], cmds, DialogUtilities.TYPE_UPLOAD, null, 0l, CommonTransitions.createDialogPulsate().copy(false), null, null, bodyCmp);
Server code
CloudinaryController class
/**
* Upload a MultipartFile as partial file.
*
* #param authToken
* #param email
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #param file
* #return "OK" if success
*/
#PostMapping("/partialUpload")
public #ResponseBody
String partialUpload(#RequestHeader(value = "authToken") String authToken, #RequestHeader(value = "email") String email, #RequestHeader(value = "partNumber") String partNumber, #RequestHeader(value = "uniqueId") String uniqueId, #RequestParam("file") MultipartFile file) throws IOException {
return cloudinaryService.partialUpload(authToken, email, partNumber, uniqueId, file);
}
/**
* Merges the files previuosly uploaded by "/partialUpload", upload that
* file to Cloudinary and returns the id assigned by Cloudinary
*
* #param authToken
* #param email
* #param uniqueId containing the totalBytes before the first "-"
* #return the id assigned by Cloudinary
*/
#GetMapping("/mergeUpload")
public #ResponseBody
String mergeUpload(#RequestHeader(value = "authToken") String authToken, #RequestHeader(value = "email") String email, #RequestHeader(value = "uniqueId") String uniqueId) throws IOException {
return cloudinaryService.mergeUpload(authToken, email, uniqueId);
}
CloudinaryService class
/**
* Upload a MultipartFile as partial file.
*
* #param authToken
* #param email
* #param partNumber
* #param uniqueId containing the totalBytes before the first "-"
* #param file
* #return "OK" if success
*/
public String partialUpload(String authToken, String email, String partNumber, String uniqueId, MultipartFile file) throws IOException {
User user = userService.getUser(authToken, email);
if (user != null) {
String output = AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber;
Path destination = Paths.get(output);
Files.copy(file.getInputStream(), destination, StandardCopyOption.REPLACE_EXISTING);
return "OK";
} else {
logger.error("Error: a not authenticated user tried to upload a file (email: " + email + ", authToken: " + authToken + ")");
return null;
}
}
/**
* Merges the files previuosly uploaded by "/partialUpload", upload that
* file to Cloudinary and returns the id assigned by Cloudinary
*
* #param authToken
* #param email
* #param uniqueId containing the totalBytes before the first "-"
* #return the id assigned by Cloudinary
*/
public String mergeUpload(String authToken, String email, String uniqueId) throws IOException {
User user = userService.getUser(authToken, email);
if (user != null) {
long totalBytes = Long.valueOf(uniqueId.split("-", 2)[0]);
List<File> files = new ArrayList<>();
int partNumber = 0;
File testFile = new File(AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber);
while (testFile.exists()) {
files.add(testFile);
partNumber++;
testFile = new File(AppApplication.uploadTempDir + "/" + uniqueId + "-" + partNumber);
}
// the list of files is ready, we can now merge them
File merged = new File(AppApplication.uploadTempDir + "/" + uniqueId);
IOCopier.joinFiles(merged, files);
// uploads the file to Cloudinary
Map uploadResult = cloudinary.uploader().upload(merged, ObjectUtils.emptyMap());
String publicId = uploadResult.get("public_id").toString();
// removes the files
for (File file : files) {
file.delete();
}
merged.delete();
return publicId;
} else {
logger.error("Error: a not authenticated user tried to upload a file (email: " + email + ", authToken: " + authToken + ")");
return null;
}
}
IOCopier class
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;
import org.apache.commons.io.IOUtils;
/**
* Useful to merge files. See: https://stackoverflow.com/a/14673198
*/
public class IOCopier {
public static void joinFiles(File destination, List<File> sources)
throws IOException {
OutputStream output = null;
try {
output = createAppendableStream(destination);
for (File source : sources) {
appendFile(output, source);
}
} finally {
IOUtils.closeQuietly(output);
}
}
private static BufferedOutputStream createAppendableStream(File destination)
throws FileNotFoundException {
return new BufferedOutputStream(new FileOutputStream(destination, true));
}
private static void appendFile(OutputStream output, File source)
throws IOException {
InputStream input = null;
try {
input = new BufferedInputStream(new FileInputStream(source));
IOUtils.copy(input, output);
} finally {
IOUtils.closeQuietly(input);
}
}
}

Answer 2

Not at this time as there's no evaluation for where the issue lies. I think the progress listener tracks the output stream writing to the upload code not the actual connection time which is normally hard to track in Java anyway.
E.g. in Java SE you would open a URL and then write to the output stream of a POST connection. Then the writing would actually occur when you try to get the input stream response. But at this point I would have no indication about the state of the upload as it's completely abstracted and happening under the hood.
So I'm not sure if this is even technically feasible.