I want to scrape data on dynamic web by the following code:
> URL<- "http://www.cbooo.cn/realtime"
> library(bitops)
> library(RCurl)
> library(XML)
> library(RSelenium)
> library(magrittr)
> checkForServer()
Warning message:
checkForServer is deprecated.
Users in future can find the function in file.path(find.package("RSelenium"), "example/serverUtils").
The sourcing/starting of a Selenium Server is a users responsiblity.
Options include manually starting a server see vignette("RSelenium-basics", package = "RSelenium")
and running a docker container see vignette("RSelenium-docker", package = "RSelenium")
> startServer()
$stop
function ()
{
tools::pskill(selPID)
}
<environment: 0x10991af0>
$getPID
function ()
{
return(selPID)
}
<environment: 0x10991af0>
Warning message:
startServer is deprecated.
Users in future can find the function in file.path(find.package("RSelenium"), "example/serverUtils").
The sourcing/starting of a Selenium Server is a users responsiblity.
Options include manually starting a server see vignette("RSelenium-basics", package = "RSelenium")
and running a docker container see vignette("RSelenium-docker", package = "RSelenium")
> remDrv <- remoteDriver()
> remDrv$browserName="Internet Explorer"
> remDrv$open()
[1] "Connecting to remote server"
Selenium message: The best matching driver provider org.openqa.selenium.ie.InternetExplorerDriver can't create a new driver instance for Capabilities [{nativeEvents=true, browserName=Internet Explorer, javascriptEnabled=true, version=, platform=ANY}]
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'DESKTOP-J0D980N', ip: '10.36.17.76', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_77'
Driver info: driver.version: unknown
Error: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
class: org.openqa.selenium.WebDriverException
Further Details: run errorDetails method
There are the following questions which I can't solve:
1 checkForServer,startServer are deprecated.
2 connecting to remote server always fials,I don't konw how to set some argurment in this funciotn and what should be done
I hope to get a anwser as soon as possible ,thanks.
The author of RSelenium provides a solution as following (https://github.com/ropensci/RSelenium/issues/81):
From Firefox 48 on-wards the gecko driver/ marionette will be needed to run Firefox with Selenium.
If you have Firefox 48 you can run the gecko driver as follows:
Refer to the guidelines
https: //developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
Download the relevant gecko driver from
https: //github.com/mozilla/geckodriver/releases
Add it to your PATH or refer to the location when starting binary (see below)
# get beta selenium standalone
RSelenium::checkForServer(beta = TRUE)
# assume gecko driver is not in our path (assume windows and we downloaded to docs folder)
# if the driver is in your PATH the javaargs call is not needed
selServ <- RSelenium::startServer(javaargs = c("- Dwebdriver.gecko.driver=\"C:/Users/john/Documents/geckodriver.exe\""))
remDr <- remoteDriver(extraCapabilities = list(marionette = TRUE))
remDr$open()
....
....
remDr$close()
selServ$stop()
Well to produce a viable working solution I would use the old version of RSelenium and everything with this code.
if (!require("XML")) {
install.packages("XML",repos= 'https://cloud.r-project.org')
library("XML")
}
#XML is a dependency
if (!require("RSelenium")) {
install.packages("https://cran.r-project.org/src/contrib/Archive/RSelenium/RSelenium_1.3.5.tar.gz", repos=NULL, type="source", dependencies = TRUE)
library("RSelenium")
}
download.file('http://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar', destfile = "~/Documents/R/library/RSelenium/bin/selenium-server-standalone.jar")
#start server
system('java -jar "~/Documents/R/library/RSelenium/bin/selenium-server-standalone.jar"')
library(RSelenium)
checkForServer()
startServer()
That is not an optimal solution. But a working solution.
Related
I'm trying to get trace logging information out of the gecko driver, in order to debug some issues we're having with moving to newer versions of Python, Robot Framework, Selenium, Firefox.
See in the DEBUG robot output below, I'm setting the capabilities to include log level arguments as part of moz:firefoxOptions.
When the capabilities are printed out for Open Browser, the moz:firefoxOptions value has been replaced with the profile info, so I never get tracing set.
20190219 13:07:22.461 - INFO - {'browserName': 'firefox', 'marionette': True, 'acceptInsecureCerts': True, 'moz:firefoxOptions': {'prefs': {'log': {'level': 'trace'}}, 'args': ['-headless', '-profile', '/tmp/tmpimda__f2']}, 'binary': '/usr/bin/firefox'}
20190219 13:07:22.461 - INFO - +--- END KW: BuiltIn.Log (1)
20190219 13:07:22.461 - INFO - +--- START KW: CustomLib.AdminClass.Open Browser [ ${LOGIN URL} | ${BROWSER} | desired_capabilities=${dc} | ff_profile_dir=${profile} ]
20190219 13:07:22.462 - INFO - Opening browser 'Firefox' to base url 'https://HOST:PORT/bmmadmin/login.jsp'.
20190219 13:07:22.480 - DEBUG - POST http://127.0.0.1:38132/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts": true, "moz:firefoxOptions": {"profile": "UEsDBBQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAdXNlci5qc6VWTW/UMBC98ytQTyCx1rbABU6lFAkJUcSq4mg59mTjrmMbe7zp/nvGyaYsbb6AWz7e89hvZt44RQjcByhfnBXBNfTGlKtZo61yDVOp9gysKAyos1fPMSR4+f5ZesoxAiGEZE/ApTBxBG2hQVGwFAzhzkThEr4rjLC7sym8F1tYtn5EETB5lhmEXE+jKlfDEblgL49Z3O3p5FoBqyM6266it9YFGFmAPqkb+6MVOJ6vmbYY3O33L9Phc1Jqcc9lFSgwjzJoj5wU56jrHPT1epiwDBnAu4BcGMPvIod7CR61s3Ew6XdiL7plmetgLFau+Wyv6NEZGCRZwMaFHZOCKHtY5XjCrAiw13ImrxFkChoPTMa+HEeAj5PTgJFZr9lSm2IyoZTO5xQzizRQqECHC1zIrCBPlNsUERSXEHBYzRNSjClHfCBp+gBhhtXpwa3IsnLYgx0JJLxnySvqVCYSumEFT0CLWk3p6I04gLo+X2++OqRcEuHNCNg11jihqDAtaRzasvlRgd1k9bXdTse6vv1yyfp2m7SjFvmadVmYsy5td1THYKnsycS6JF/MYcmSOo8cB7uyJPxIpfagAEJlZQBB0k4/6RDxkrSQI230YEOihPaZVFtoiqeMWphGhKV2CiLI6lgWc1BTxuRzc/f2PuajEGO2DiSnJBXayi/J27gMIlYzUSowhskK5O4jlCIZ/ND9mabRDImMjm1v7JVxceYoJ+gbyvgwmBQRnXdmVSsQBqvunS1ytgl+8rlVrv+R3Ucvc0F9T3+/feO22/xRdp7+F/vwzmh5YPnjJhW1bvP8v/xv7cfL3w09tAzs0TkTGTmEC8eNT15f8ugjB2sNtG1rRXPGbnlWfjhG9gjbFm5roh878kYSO7vu+XoUXhgnd0ZHnK6IE0Zb4Feu9mTthTZ5AFq9rdAcZqlLLPwp3JJ/l4fb0UY6GfxTqvajvnP51dEHVzRiMcXhlXtKhUjXtkrH6rDKvz3NxJUBu8VsCRdv3/5J65emqUX5MMY1vDhw1XnC4N48WYzHmLub104Js/DmUWoD9MRc0NQWvKvRfJmawUcMWiJ/TBsKlW+NlqywhrqAsGlfR+TKZb7TyI6y9eP0WMqDB+8pCIYiYDiwTor25Bdz2EmRnsID3LXjbHAn916T7VO2WD4y1QS5PzF+Jh3GIvymNJWmMLmLBhi/AFBLAQIUAxQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAAAAAAAAAAACkgQAAAAB1c2VyLmpzUEsFBgAAAAABAAEANQAAAKMDAAAAAA=="}}}, "desiredCapabilities": {"browserName": "firefox", "acceptInsecureCerts": true, "moz:firefoxOptions": {"profile": "UEsDBBQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAdXNlci5qc6VWTW/UMBC98ytQTyCx1rbABU6lFAkJUcSq4mg59mTjrmMbe7zp/nvGyaYsbb6AWz7e89hvZt44RQjcByhfnBXBNfTGlKtZo61yDVOp9gysKAyos1fPMSR4+f5ZesoxAiGEZE/ApTBxBG2hQVGwFAzhzkThEr4rjLC7sym8F1tYtn5EETB5lhmEXE+jKlfDEblgL49Z3O3p5FoBqyM6266it9YFGFmAPqkb+6MVOJ6vmbYY3O33L9Phc1Jqcc9lFSgwjzJoj5wU56jrHPT1epiwDBnAu4BcGMPvIod7CR61s3Ew6XdiL7plmetgLFau+Wyv6NEZGCRZwMaFHZOCKHtY5XjCrAiw13ImrxFkChoPTMa+HEeAj5PTgJFZr9lSm2IyoZTO5xQzizRQqECHC1zIrCBPlNsUERSXEHBYzRNSjClHfCBp+gBhhtXpwa3IsnLYgx0JJLxnySvqVCYSumEFT0CLWk3p6I04gLo+X2++OqRcEuHNCNg11jihqDAtaRzasvlRgd1k9bXdTse6vv1yyfp2m7SjFvmadVmYsy5td1THYKnsycS6JF/MYcmSOo8cB7uyJPxIpfagAEJlZQBB0k4/6RDxkrSQI230YEOihPaZVFtoiqeMWphGhKV2CiLI6lgWc1BTxuRzc/f2PuajEGO2DiSnJBXayi/J27gMIlYzUSowhskK5O4jlCIZ/ND9mabRDImMjm1v7JVxceYoJ+gbyvgwmBQRnXdmVSsQBqvunS1ytgl+8rlVrv+R3Ucvc0F9T3+/feO22/xRdp7+F/vwzmh5YPnjJhW1bvP8v/xv7cfL3w09tAzs0TkTGTmEC8eNT15f8ugjB2sNtG1rRXPGbnlWfjhG9gjbFm5roh878kYSO7vu+XoUXhgnd0ZHnK6IE0Zb4Feu9mTthTZ5AFq9rdAcZqlLLPwp3JJ/l4fb0UY6GfxTqvajvnP51dEHVzRiMcXhlXtKhUjXtkrH6rDKvz3NxJUBu8VsCRdv3/5J65emqUX5MMY1vDhw1XnC4N48WYzHmLub104Js/DmUWoD9MRc0NQWvKvRfJmawUcMWiJ/TBsKlW+NlqywhrqAsGlfR+TKZb7TyI6y9eP0WMqDB+8pCIYiYDiwTor25Bdz2EmRnsID3LXjbHAn916T7VO2WD4y1QS5PzF+Jh3GIvymNJWmMLmLBhi/AFBLAQIUAxQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAAAAAAAAAAACkgQAAAAB1c2VyLmpzUEsFBgAAAAABAAEANQAAAKMDAAAAAA=="}, "binary": "/usr/bin/firefox", "marionette": true}}
Anyone have any ideas what I may be doing wrong here? And/OR some alternative methods of getting trace logging from geckodriver?
BTW: the moz:firefoxOptions I'm setting may not be correct, they're just the latest combination I have tried, but I can't tell what is going to work until the options I set are actually being sent to firefox.
Environment:
Python 3.6.3
geckodriver 0.24.0 ( 2019-01-28)
Mozilla Firefox 60.4.0
PIP:
requests (2.21.0)
robotframework (3.1.1)
robotframework-requests (0.5.0)
robotframework-selenium2library (3.0.0)
robotframework-seleniumlibrary (3.3.1)
selenium (3.141.0)
setuptools (40.8.0)
urllib3 (1.24.1)
ROBOT:
Open Browser To Login Page
${dc}= CustomLib.Get Desired Capabilities ${BROWSER}
Set To Dictionary ${dc}
... binary ${PATH TO MARIONETTE}
Log ${dc}
${profile}= CustomLib.Create FF Profile
Open Browser ${LOGIN URL} ${BROWSER} desired_capabilities=${dc} ff_profile_dir=${profile}
PYTHON:
#keyword
def get_desired_capabilities(browser):
_capabilities = {'ff' : DesiredCapabilities.FIREFOX,
'firefox' : DesiredCapabilities.FIREFOX
}
browserKey = str(browser).lower()
desired = _capabilities[browserKey]
BuiltIn().log("get_desired_capabilities B4 :" + str(desired))
#THIS IS ASSUMING FIREFOX
desired['marionette'] = True
desired['acceptSslCerts'] = True
desired['ignoreProtectedModeSettings'] = True
desired['headless'] = True
desired['moz:webdriverClick'] = True
desired['moz:firefoxOptions'] = {'log':{'level':'trace'}, 'args':['-headless']}
#desired['moz:firefoxOptions']['log'] = {'level':'trace'}
#desired['moz:firefoxOptions']['args'] = ['-headless']
BuiltIn().log("get_desired_capabilities returning :" + str(desired))
return desired
#keyword
def create_ff_download_profile(path):
from selenium import webdriver
BuiltIn().log('PATH: ' + path)
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.helperApps.alwaysAsk.force", False);
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", 'text/xml,text/plain,application/octet-stream')
fp.set_preference("browser.download.dir", path)
fp.update_preferences()
return fp.path
thanks,
Tom
This is how I set the log level to be trace in Firefox:
*** Settings ***
Library SeleniumLibrary
*** Keywords ***
${options}= Evaluate sys.modules['selenium.webdriver.firefox.options'].Options() sys
${options.log.level}= Set Variable trace
${driver}= Create Webdriver Firefox options=${options}
Go To https://www.google.com
It's basically following this python pattern from the firefox docs and translating it into Robotframework
Hope this helps!
Below is my code and getting this error only some times. Surprisingly, when i run the same code line by line, it is working as expected!
from selenium import webdriver
browser=webdriver.Chrome()
browser.maximize_window()
browser.get('https://www.gmail.com')
email=browser.find_element_by_xpath('//*[#id="identifierId"]')
email.send_keys('my email-id')
next=browser.find_element_by_xpath('//*[#id="identifierNext"]/content/span')
next.click()
browser.implicitly_wait(9)
passwd=browser.find_element_by_xpath('//*[#id="password"]/div[1]/div/div[1]/input')
passwd.send_keys('mypassword')
next1=browser.find_element_by_xpath('//*[#id="passwordNext"]/content/span')
next1.click()
Error:
selenium.common.exceptions.WebDriverException: Message: unknown error: Element <span class="RveJvd snByac">...</span> is not clickable at point (942, 426). Other element would receive the click: <div class="
ANuIbb IdAqtf" jsname="k4HEge" tabindex="0"></div>
(Session info: chrome=61.0.3163.91)
(Driver info: chromedriver=2.32.498550 (9dec58e66c31bcc53a9ce3c7226f0c1c5810906a),platform=Windows NT 6.1.7601 SP1 x86_64)
Hope Your Problem will be solved
from selenium import webdriver
browser=webdriver.Chrome()
browser.maximize_window()
browser.get('https://www.gmail.com')
email=browser.find_element_by_xpath('//*[#id="identifierId"]')
email.send_keys('my email-id')
next=browser.find_element_by_xpath('//*[#id="identifierNext"]/content/span')
next.click()
passwd=browser.find_element_by_name("password")
passwd.send_keys('mypassword')
next1=browser.find_element_by_xpath('//*[#id="passwordNext"]/content/span')
next1.click()
Below code should work out for you. The basic idea is to wait for the password field until its ready in DOM
from selenium import webdriver
browser=webdriver.Chrome()
browser.maximize_window()
browser.get('https://www.gmail.com')
email=browser.find_element_by_xpath('//*[#id="identifierId"]')
email.send_keys('my email-id')
next=browser.find_element_by_xpath('//*[#id="identifierNext"]/content/span')
next.click()
passwd=browser.find_element_by_xpath("//input[#name='password']")
wait = WebDriverWait(driver, 20)
wait.until(EC.element_to_be_clickable(passwd))
passwd.send_keys('mypassword')
next1=browser.find_element_by_xpath('//*[#id="passwordNext"]/content/span')
next1.click()
I've been using Selenium webdriver for quite a while. Recently I came across this page: https://www.myagedcare.gov.au/service-finder
It seem very simple and I try to use id locator: bylocation
However, no matter how long I wait for the element to be ready by adding wait for some time, or wait for element to be ready like:
WebDriver driver = getDriver();
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(
ExpectedConditions.elementToBeClickable(By.id("byname")))
.click();
the element is never ready, and I always get error like "no such element: Unable to locate element:"
What might be going on? I've used locator for quite a while and I do not know how this can happen.
I am using chromedriver 2.30 and Chrome browser v.60.
A simple id locator like this in www.google.com does not give me any trouble. I tested it so that I do not think it's driver or browser issue.
Logs
net.serenitybdd.core.exceptions.SerenityWebDriverException: Timed out after 10 seconds waiting for visibility of element located by By.xpath: //input[#id='bylocation']
Build info: version: '2.46.0', revision: '61506a4624b13675f24581e453592342b7485d71', time: '2015-06-04 10:22:50'
System info: host: 'yun-PC', ip: '192.168.1.14', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_71'
Caused by: org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//input[#id='bylocation']"}
(Session info: chrome=60.0.3112.50)
(Driver info: chromedriver=2.30.477700 (0057494ad8732195794a7b32078424f92a5fce41),platform=Windows NT 6.1.7601 SP1 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 20.04 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Try this xpath and let me know if it works,
//div[#id='edit-search-by--2']//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']
Also try waiting for visibility of element and not for element to be clickable.
Edit:
The above Xpath is perfect. Problem was that the element was located in a frame.
Below code will get you going with the click part:
driver.get("https://www.myagedcare.gov.au/service-finder?tab=help-at-home");
new WebDriverWait(driver, 15).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//iframe[#id='content']")));
driver.switchTo().frame(driver.findElement(By.xpath("//iframe[#id='content']")));
new WebDriverWait(driver, 15).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']")));
WebElement location = driver.findElement(By.xpath("//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']"));
WebElement name = driver.findElement(By.xpath("//label[normalize-space(text())='Name']/preceding-sibling::input[#type='radio']"));
location.click();
Thread.sleep(3000);
name.click();
driver.switchTo().parentFrame();
Click working video - https://www.screencast.com/t/0Fsw7gGZ
As the Radio Buttons are inside input tags, instead of id use xpath locator as follows:
WebDriverWait wait6 = new WebDriverWait(driver, 10);
WebElement radio_name = wait6.until(ExpectedConditions.elementToBeClickable(By.xpath("//div[#id='byname']/input[#id='byname']")));
radio_name.click();
Let me know if this Answers your Question.
I'm using a ruby/cucumber/capybara framework with versions:
capybara (2.10.1)
selenium-webdriver (3.0.0)
ruby 2.3.1p112 (2016-04-26 revision 54768) [x64-mingw32]
and I'm trying to register a new driver with some settings passed in the profile to capybara. My code looks like this:
Capybara.register_driver :debug do |app|
profile = Selenium::WebDriver::Firefox::Profile.new
Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile)
end
and I've tried as well with:
Capybara::Selenium::Driver.new(app, :profile => profile)
Then I just use the following to select that driver:
Capybara.default_driver = :debug
but in both cases, when I try to run any test, I get the following error:
ArgumentError: unknown option: {:profile=>#<Selenium::WebDriver::Firefox::Profile:0x000000068ca798 #model=nil, #native_events=true, #secure_ssl=false, #untrusted_issuer=true, #load_no_focus_lib=false, #additional_prefs={}, #extensions={}>}
Any idea what the problem could be? and how to ammend it?
Firefox Profiles are not yet supported in Ruby bindings for geckodriver, which is needed for Firefox 48+. Watch this issue for (hopefully very soon) resolution: https://github.com/SeleniumHQ/selenium/issues/2933
I have a small, but growing (hopefully) suite of integration tests that I've built using Nightwatch.js (v0.5.6) on top of the Selenium webdriver (v2.41.0). I've always gotten the occasional Element not found in the cache error, but I'm working on a large validation test case - a file that contains over 2 dozen individual tests plus setUp(). I haven't been able to finish running this test yet and that's a problem.
There was an error while executing the Selenium command - enabling the --verbose option might offer more details.
Element not found in the cache - perhaps the page has changed since it was looked up
The error always seems to occur right at the end of my setUp() function, but I can't find the answer that will prevent this caching from happening. Here's my setUp() function:
setUp: function(browser) {
console.log('Logging in & navigating to Eligibility Groups...');
login(browser, app.masterAdminUsername, app.masterAdminPassword)
// Navigate to Eligibility Groups
.waitForElementVisible('button[data-action="EligibilityGroups"]', 1000, function() {
browser
.click('button[data-action="EligibilityGroups"]', function() {
console.log('Link clicked. Waiting for #btnCreate to be visible');
browser
.waitForElementVisible('#btnCreate', 1000, function() {
console.log('Exiting setUp()');
});
});
})
}
Eventually, I get this:
There was an error while executing the Selenium command - enabling the --verbose option might offer more details.
Element not found in the cache - perhaps the page has changed since it was looked up
Command duration or timeout: 3.72 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
Build info: version: '2.41.0', revision: '3192d8a', time: '2014-03-27 17:17:32'
System info: host: 'robwilkerson.local', ip: '172.20.1.112', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9.4', java.version: '1.6.0_65'
Session ID: 04f47f5c-fda0-f049-9ec1-1d3a40ac44fe
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{platform=MAC, acceptSslCerts=true, javascriptEnabled=true, browserName=firefox, rotatable=false, locationContextEnabled=true, version=30.0, cssSelectorsEnabled=true, databaseEnabled=true, handlesAlerts=true, browserConnectionEnabled=true, nativeEvents=false, webStorageEnabled=true, applicationCacheEnabled=true, takesScreenshot=true}]
✖ Timed out while waiting for element <#btnCreate> to be visible for 1000 milliseconds. - expected "visible" but got: not visible
The test case varies, but it always seems to fail "while waiting for element <#btnCreate> to be visible for 1000 milliseconds". Changing the number of ms I wait only changes the number of ms reported in the error.
What can I do here? Is there something wrong with my scripts? Everything I've read and everything I've tried has gotten me nowhere.
In case anyone drops by, what I've found through trial and error is:
Virtually any time a .click() loads or reloads content, I need to add a .pause(). Usually, .pause(1000) is sufficient, but longer is occasionally necessary. Using .waitForElementVisible() alone rarely works consistently.
Whenever I do use .waitForElementVisible(), I've used a default timeout of 30000. There doesn't seem to be any penalty for larger values.
There may be better answers, but this has worked reasonably well for me so far.
I'm marking this as the answer for now, but am open to changing that if someone has a better strategy.