I'd like to gather some info on Google Chrome when it's running. Currently I'm doing this using some applescript (https://gist.github.com/jcla1/6525572), but I'd like to rewrite this in C or Go.
Does anyone know of an API Chrome exposes to gather information like:
# open tabs
current active URL
So far I've only found out that I can get the current tab's title using CGWindowListCopyWindowInfo (Carbon), which works well but obviously doesn't provide the URL (and not the # open tabs, etc.).
Ideally Chrome would have a event architecture which I could hook into and receive all the events to do with entering a new URL.
P.S. Only required to work on OSX!
I have written Go programs to remotely control/inspect chrome using the Remote Debugging Protocol. The way it works is you first fetch JSON data from a specific url and it gives the open tabs and basic information such as the tab's current url. You can then use the websocket links in the JSON object to contorl/inspect individual tabs.
Since you only want basic information, you can ignore most of the debugging api and just download the JSON index. Frist, start chrome using chrome --remote-debugging-port=9222. Note that all chrome windows need to be closed for this to work. You may also use --user-data-dir=<some dir> to launch with a fresh profile so you can leave your other chrome windows open.
Once you have the port open, obtaining the data is easy. Just go to http://localhost:9222/json and unmarshal the data. Here is the code I used :
type Tab struct {
Description string `json:"description"`
DevtoolsFrontendUrl string `json:"devtoolsFrontendUrl"`
FaviconUrl string `json:"faviconUrl"`
Id string `json:"id"`
ThumbnailUrl string `json:"thumbnailUrl"`
Title string `json:"title"`
Type string `json:"type"`
Url string `json:"url"`
WebSocketDebuggerUrl string `json:"webSocketDebuggerUrl"`
}
func GetTabs() ([]Tab, error) {
resp, err := http.Get("http://localhost:9222/json")
if err != nil {
return nil, err
}
defer resp.Body.Close()
var tabs []Tab
err = json.NewDecoder(resp.Body).Decode(&tabs)
if err != nil {
return nil, err
}
return tabs, nil
}
You can obtain more information here.
There is unfortunately no way to receive an event when a new tab is opened using this API. However, you can get notified when a tab you are connected to with websockets changes in any way. If you are willing to go crazy, you can build an extension to monitor for changes such as new tabs and loaded urls.
Related
Background story: I need to obtain the handles of the tagged Twitter users from an attached Twitter media. There's no current API method to do that unfortunately (see https://twittercommunity.com/t/how-to-get-tags-of-a-media-in-a-tweet/185614 and https://github.com/twitterdev/open-evolution/issues/34).
I have no other choice but to scrape, this is an example URL: https://twitter.com/justinwood_/status/1626275168157851650/media_tags. This is the page which pops up when you click on the tags link under the media of the parent Tweet: https://twitter.com/justinwood_/status/1626275168157851650/
The React generated DOM is deep and ugly, but would be scrapeable, however I do not want to log in with any account to get banned. Unfortunately when you visit https://twitter.com/justinwood_/status/1626275168157851650/media_tags in an Incognito window the popup shows up dead empty. However when I dig into the network requests the /TweetDetail GraphQL endpoint is full of messages about the anonymous page visit, fortunately it still contains the list of handles I need despite of all of this.
So what I need to have is a scraper which is able to process JavaScript, and capture the response for that specific GraphQL call. Selenium uses a headless Chrome under the hood, so it is able to process JavaScript, and Selenium-Wire offers the ability to capture the response.
Unfortunately my crafted Selenium-Wire script only has the TweetResultByRestId and UsersByRestId GraphQL requests but is missing the TweetDetail. I don't know what to tweak to make all the requests to happen. I iterated over a ton of Chrome options. Here is a variation of my script:
from seleniumwire import webdriver
from selenium.webdriver.chrome.service import Service
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-gpu")
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--headless") # for Jenkins
chrome_options.add_argument("--disable-dev-shm-usage") # Jenkins
chrome_options.add_argument('--start-maximized')
chrome_options.add_argument('--window-size=1900,1080')
chrome_options.add_argument('--ignore-certificate-errors-spki-list')
chrome_options.add_argument('--ignore-ssl-errors')
selenium_options = {
'request_storage_base_dir': '/tmp', # Use /tmp to store captured data
'exclude_hosts': ''
}
ser = Service('/usr/bin/chromedriver')
ser.service_args=["--verbose", "--log-path=test.log"]
driver = webdriver.Chrome(service=ser, options=chrome_options, seleniumwire_options=selenium_options)
tweet_id = "1626275168157851650"
twitter_media_url = f"https://twitter.com/justinwood_/status/{tweet_id}/media_tags"
driver.get(twitter_media_url)
driver.wait_for_request("/TweetDetail", timeout=10)
Any ideas?
Apparently it looks like I'd rather need to scrape the parent Tweet URL https://twitter.com/justinwood_/status/1626275168157851650/ and right now it seems my craved GraphQL call happens. Probably I got confused while trying 100 combinations.
I am opening a webpage in the cefsharp browser and trying to send a set of JSON data to my website's .aspx page along with query string. While the query string is not an issue but sending the JSON data to the same URL is what I am trying to fix. Earlier I was using Window's native WebBrowser control's Navigate method where I was passing the URL along with query string as well as a byte array. But, I don't find any such method to post the data. Various discussion and posts regarding the same don't have a clear example. Can you provide a sample code/example to show how to achieve that? Here is the code I've been using:
ChromiumWebBrowser browser = new ChromiumWebBrowser();
browser.Address = "https://webhook.site";
browser.Width = System.Windows.SystemParameters.PrimaryScreenWidth;
browser.Height = System.Windows.SystemParameters.PrimaryScreenHeight;
browser.RequestHandler = this;
browser.IsBrowserInitializedChanged += (sender, args) =>
{
if (browser.IsBrowserInitialized)
{
browser.LoadUrlWithPostData("https://webhook.site/#/cba9d04b-01ff-40ef-b223-0917d127ecbe/6ce82e34-28df-4900-88ef-c932a446c6b0/1", Encoding.UTF8.GetBytes("test=123&data=456"));
}
};
I'm implementing Offline Playback with HLS Fairplay following the demo in the FairPlay Streaming Server SDK v4.0.1 that uses AVContentSessionKey.
I download three contents, each content is downloaded and persisted correctly, both the .movpkg and its content key on the documents directory, when I turn off the WIFI these three contents downloaded plays correctly without any problems, before playing Im using this code:
let urlAsset = element.urlAsset!
ContentKeyManager.shared.contentKeySession.addContentKeyRecipient(urlAsset)
if !urlAsset.resourceLoader.preloadsEligibleContentKeys {
urlAsset.resourceLoader.preloadsEligibleContentKeys = true
}
self.present(playerViewController, animated: true, completion: {
AssetPlaybackManager.sharedManager.setAssetForPlayback(urlAsset)
})
So far so good. But the problem is when I close the application (Home button to close applications) and then play the downloaded contents, only the last content downloaded plays correctly, the other ones (first and second) send these error on the console.
Error Domain=AVFoundationErrorDomain Code=-11800 "The operation could not be completed"
UserInfo={NSUnderlyingError=0x1c065d760 {Error Domain=NSOSStatusErrorDomain Code=-16227 "(null)"},
NSLocalizedFailureReason=An unknown error occurred (-16227),
NSURL=file:///private/var/mobile/Containers/Data/Application/A950D8DB-B711-47E3-AAF5-C95CC9682430/Library/com.apple.UserManagedAssets.kkG8Ih/644986_7798B8476A473F68.movpkg/, NSLocalizedDescription=The operation could not be completed}
I double check the .movpkg with the keys in the documents directory and appears correctly
/Documents/.keys/one-key
/Documents/.keys/two-key
/Documents/.keys/three-key
Before the error occurs the ContentKeyDelegate is called and the key is loaded and passed to the request correctly.
if persistableContentKeyExistsOnDisk(withContentKeyIdentifier: assetIDString) {
let urlToPersistableKey = urlForPersistableContentKey(withContentKeyIdentifier: assetIDString)
guard let contentKey = FileManager.default.contents(atPath: urlToPersistableKey.path) else {
/
pendingPersistableContentKeyIdentifiers.remove(assetIDString)
return
}
/
Create an AVContentKeyResponse from the persistent key data to use for requesting a key for
decrypting content.
*/
let keyResponse = AVContentKeyResponse(fairPlayStreamingKeyResponseData: contentKey)
/
keyRequest.processContentKeyResponse(keyResponse)
return
}
If I print the contentKeyRecipients the three contents appears correctly
- (lldb) po
ContentKeyManager.shared.contentKeySession.contentKeyRecipients ▿ 3
elements
- 0 : AVURLAsset: 0x1c0234d40, URL = file:///private/var/mobile/Containers/Data/Application/E791A4DE-4261-46B7-A84D-D10B27035FAE/Library/com.apple.UserManagedAssets.kkG8Ih/539628_20469336224AA388.movpkg
- 1 : AVURLAsset: 0x1c0234fa0, URL = file:///private/var/mobile/Containers/Data/Application/E791A4DE-4261-46B7-A84D-D10B27035FAE/Library/com.apple.UserManagedAssets.kkG8Ih/644986_7798B8476A473F68.movpkg
- 2 : AVURLAsset: 0x1c42391c0, URL = file:///private/var/mobile/Containers/Data/Application/E791A4DE-4261-46B7-A84D-D10B27035FAE/Library/com.apple.UserManagedAssets.kkG8Ih/573744_62377F9549C45B93.movpkg
My tests are in iOS 11.1.2 and iOS 11.2 beta 2
I'm not sure what is happening, but seems to be a problem with the persisted key, I don't how if each content needs to be associated with one AVContentKeySession at time.
If someone faced a similar problem, any help would be appreciated.
Thanks in advance
I'm having similar issue.
however, since I need to support iOS 10, I'm not using the new AVContentKeyResponse class. Instead, I'm loading the persistent content key myself, and pass it to the loading request.
Anyway, I'm getting exact the same error as you and same behavior. One thing to note is that if I remove the code that loads persistent content key from disk, and always fetch the key from server, then everything works. But this defeats the purpose of "offline" playback...
So it seems like the system thinks the persistent content key is invalid...
Which TLLV you used on the server side to specify Rental Duration of the downloaded content? Did you use Content key duration TLLV or Offline Key TLLV? If you used Offline Key TLLV you need to double check that "Content ID" field is different for every downloaded movie.
We had encounter this error message, too.
It will happen when content exceed over expiration date which set in server side.
For example :
We give 10 minutes of expiration date for Video A
Download this Video A, and verify CKC delivery correctly (print log)
Play Video A without connection
Take a break (after 11 minutes), close App, and launch App again, select Video A to Play
Show below error message from AVPlayerItem.error.description:
Error Domain=AVFoundationErrorDomain Code=-11800
"The operation could not be completed" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-16227),
NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x1d4257310
{Error Domain=NSOSStatusErrorDomain Code=-16227 "(null)"}}
You can refresh encrypted data again by
AVAssetResourceLoaderDelegate
or use AVContentSessionKey
Reference : https://developer.apple.com/videos/play/wwdc2018/507/
Make sure you set correct offline content identifier on a serverside. The identifier you set should be associated with the specific rendition/stream allowed by the license. This helped me.
Good day, I'm working on a Servlet that must return a PDF file and the message log for the processing done with that file.
So far I'm passing a boolean which I evaluate and return either the log or the file, depending on the user selection, as follows:
//If user Checked the Download PDF
if (isDownload) {
byte[] oContent = lel;
response.setContentType("application/pdf");
response.addHeader("Content-disposition", "attachment;filename=test.pdf");
out = response.getOutputStream();
out.write(oContent);
} //If user Unchecked Download PDF and only wants to see logs
else {
System.out.println("idCompany: "+company);
System.out.println("code: "+code);
System.out.println("date: "+dateValid);
System.out.println("account: "+acct);
System.out.println("documentType: "+type);
String result = readFile("/home/gianksp/Desktop/Documentos/Logs/log.txt");
System.setOut(System.out);
// Get the printwriter object from response to write the required json object to the output stream
PrintWriter outl = response.getWriter();
// Assuming your json object is **jsonObject**, perform the following, it will return your json object
outl.print(result);
outl.flush();
}
Is there an efficient way to return both items at the same time?
Thank you very much
HTTP protocol doesn't allow you to send more than one HTTP response per one HTTP request. With this restriction in mind you can think of the following alternatives:
Let client fire two HTTP requests, for example by specifyingonclick event handler, or, if you returned HTML page in the first response, you could fire another request on window.load or page.ready;
Provide your for an opportunity of choosing what he'd like to download and act in a servlet accordingly: if he chose PDF - return PDF; if he chose text - return text and if he chose both - pack them in an archive and return it.
Note that the first variant is both clumsy and not user friendly and as far as I'm concerned should be avoided at all costs. A page where user controls what he gets is a much better alternative.
You could wrap them in a DTO object or place them in the session to reference from a JSP.
I am a new bie on GWT, I wrote an application on abc.com, I have another application i.e. xyz.com, xyz.com?id=1 provides me a data in json format, I was thinking to find a way that how to get that json file in abc.com via RPC call, because I have seen tutorials in which RPC calls are used to get data from its server. any help will be appreciated.
EDIT
I am trying to implement this in this StockWatcher tutorial
I changed my code slightly change to this
private static final String JSON_URL = "http://localhost/stockPrices.php?q=";
AND
private void refreshWatchList() {
if (stocks.size() == 0) {
return;
}
String url = JSON_URL;
// Append watch list stock symbols to query URL.
Iterator iter = stocks.iterator();
while (iter.hasNext()) {
url += iter.next();
if (iter.hasNext()) {
url += "+";
}
}
url = URL.encode(url);
MyJSONUtility.makeJSONRequest(url, new JSONHandler() {
#Override
public void handleJSON(JavaScriptObject obj) {
if (obj == null) {
displayError("Couldn't retrieve JSON");
return;
}
updateTable(asArrayOfStockData(obj));
}
});
}
before when I was requesting my url via RequestBuilder it was giving me an exception Couldn't retrieve JSON but now JSON is fetched and status code is 200 as I saw that in firebug but it is not updating on table. Kindly help me regarding this.
First, you need to understand the Same Origin Policy which explains how browsers implement a security model where JavaScript code running on a web page may not interact with any resource not originating from the same web site.
While GWT's HTTP client and RPC call can only fetch data from the same site where your application was loaded, you can get data from another server if it returns json in the right format. You must be interacting with a JSON service that can invoke user defined callback functions with the JSON data as argument.
Second, see How to Fetch JSON DATA