How to test Android toast messages in Appium ( selenium Java) - selenium-webdriver

Question

I am using Selenium with Java to run scripts on android (thru Appium server).
I see that it is not possible to locate a toast by using selenium's
driver.findElement(By.LinkText("User not logged in")
in Appium
But can be used in Selendroid to capture toast messages.
I there a way I can use both Selendroid and Appium in the same script?

Answer 1

Finally, we are able to read the toast message without the need of taking screenshots and performing OCR.
I have tested this on Appium 1.15.1.
Toast messages comes under com.package.system.
Normally, Xpath for this will be "/hierarchy/android.widget.Toast".
And, Class Name will be "android.widget.settings"
You can confirm this by refreshing element inspector screen when toast message is displayed.
WebDriverWait waitForToast = new WebDriverWait(driver.25);
waitForToast.until(ExpectedConditions.presenceOfElementLoacted(By.xpath("/hierarchy/android.widget.Toast")));
String toastMessage = driver.findElement((By.xpath("/hierarchy/android.widget.Toast")).getText();
System.out.println(toastMessage);

Answer 2

Method 1: from Appium version 1.6.4 supports toast messages, for that you need to use automationName:'uiautomator2'.
toast = driver.find_element(:xpath, "//android.widget.Toast[1]")
if toast.text == "Hello"
But i don't recommend this because uiautomator2 is not stable yet.
Method 2:
Trigger text message on the screen
Capture screenshots
Convert image to text file
def assettoast(string)
sname = (0...8).map { (65 + rand(26)).chr }.join
$driver.driver.save_screenshot("#{sname}")
# Make sure tesseract is installed in the system. If not you can install using "brew install tesseract" in mac
system ("tesseract #{sname} #{sname}")
text_file="#{sname}.txt"
var= get_string_from_file(string, text_file)
raise if var != true
end
Check whether toast message is there in text file
def get_string_from_file(word, filename)
File.readlines(filename).each do |line|
return true if line.include?(word)
end
end

Answer 3

Looks like you can't switch driver type within same session.
If you trying to switch to Selendroid only for toast verification - you could use OSR image recognition engine.
Check this answer w/ Ruby bindings
Idea is quite simple:
make toast message to appear
take few screenshots
iterate over taken screenshots and look for required text
Here is nice and simple example of OCR usage in Java: tess4j example (make sure that Tesseract engine installed)

Answer 4

Step 1:
File scrFile=null;
String path1 = null;
BufferedImage originalImage=null;
BufferedImage resizedImage=null;
System.out.println("Starting\n\n\n\n");
scrFile = ((TakesScreenshot) appiumDriver).getScreenshotAs(OutputType.FILE);
System.out.println("after scrfile\n\n\n\n");
originalImage = ImageIO.read(scrFile);
System.out.println("after originalFile\n\n\n");
BufferedImage.TYPE_INT_ARGB : originalImage.getType();
resizedImage = CommonUtilities.resizeImage(originalImage, IMG_HEIGHT, IMG_WIDTH);
ImageIO.write(resizedImage, "jpg", new File(path + "/"+ testCaseId + "/img/" + index + ".jpg"));
Image jpeg = Image.getInstance(path + "/" + testCaseId + "/img/"+ index + ".jpg");
Step 2:
BufferedImage pathforToast= original image;
Step 3:
System.setProperty("jna.library.path","C:/Users/Dell/workspace/MOBILEFRAMEWORK/dlls/x64/");
Tesseract instance = Tesseract.getInstance();
`enter code here`ImageIO.scanForPlugins();
String result=null;
result = instance.doOCR(pathforToast);`enter code here`
System.out.println(result);`enter code here`

Answer 5

Appium 1.6.4#beta latest version supports toast messages

Answer 6

Take screen shot of Toast Message page and try to convert the image file in to Text and verify the text using the below code.
public void imageconversion(String filePath) throws IOException,
{
ITesseract instance = new Tesseract();
//file path is the image which you need to convert to text
File imageFile = new File(filePath);
BufferedImage img = null;
img = ImageIO.read(imageFile);
BufferedImage blackNWhite = new BufferedImage(img.getWidth(),img.getHeight(), BufferedImage.TYPE_BYTE_BINARY);
Graphics2D graphics = blackNWhite.createGraphics();
graphics.drawImage(img, 0, 0, null);
//path where your downloaded tessdata exists
instance.setDatapath("E://ocr//data");
//What language you required to convert,( e.g. English)
instance.setLanguage("eng");
String result = instance.doOCR(blackNWhite);
System.out.println(result);
}

Answer 7

Appium Directly does not give any API to read toast message we need to do it using tess4j jar. First we need to take screen shot and then we need to read the text from screen shot using tess4j API.
static String scrShotDir = "screenshots";
File scrFile;
static File scrShotDirPath = new java.io.File("./"+ scrShotDir+ "//");
String destFile;
static AndroidDriver driver = null;
public String readToastMessage() throws TesseractException {
String imgName = takeScreenShot();
String result = null;
File imageFile = new File(scrShotDirPath, imgName);
System.out.println("Image name is :" + imageFile.toString());
ITesseract instance = new Tesseract();
File tessDataFolder = LoadLibs.extractTessResources("tessdata"); // Extracts
// Tessdata
// folder
// from
// referenced
// tess4j
// jar
// for
// language
// support
instance.setDatapath(tessDataFolder.getAbsolutePath()); // sets tessData
// path
result = instance.doOCR(imageFile);
System.out.println(result);
return result;
}
/**
* Takes screenshot of active screen
*
* #return ImageFileName
*/
public String takeScreenShot() {
File scrFile = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
SimpleDateFormat dateFormat = new SimpleDateFormat("dd-MMM-yyyy__hh_mm_ssaa");
new File(scrShotDir).mkdirs(); // Create folder under project with name
// "screenshots" if doesn't exist
destFile = dateFormat.format(new Date()) + ".png"; // Set file name
// using current
// date time.
try {
FileUtils.copyFile(scrFile, new File(scrShotDir + "/" + destFile)); // Copy
// paste
// file
// at
// destination
// folder
// location
} catch (IOException e) {
System.out.println("Image not transfered to screenshot folder");
e.printStackTrace();
}
return destFile;
}
For more details Refer this video - https://www.youtube.com/watch?v=lM6-ZFXiSls

Answer 8

I found three ways to capture a Toast message and verify them.
To get the page source and verify the toast message from it.
public void verifyToastMessageUsingPageSource(String toastmsg) throws InterruptedException {
boolean found = false;
for(int i =0 ; i <8; i++){
if(getDriver().getPageSource().contains("class=\"android.widget.Toast\" text=\""+toastmsg+"\"")){
found = true;
break;
}
Thread.sleep(300);
}
Assert.assertTrue(found,"toast message "+toastmsg+" is present");
}
Similar can be found using Xpath: //android.widget.Toast[1]
Using the grep command, wait for a toast message in uiautomator events.
run the command before clicking and toast message will be varified.
adb shell uiautomator events | grep "ToastMessgae"
This is tricky and needs more code to run.
start capturing screenshots thread.
perform click action
stop screen capturing thread.
extract text from the captured images using OCR and verify the toast message is present in the captured images.
I prefer the 1st and 2nd option, it provides validation in less time with less code.
comment if you need code for 2nd and 3rd point.

Answer 9

Appium with version number>=1.6.4 supports toast notification with UiAutomator2.
In Javascript with webdriver you can do like this
let toast=await driver1.elements("xpath","/hierarchy/android.widget.Toast");
let data=await toast[0].text();
console.log(data)