RSelenium: Scraping links on page - rselenium

I'm relatively new to RSelenium. I have successfully managed to log into a site from where I need to pull all web links.
That overview page looks like this:
<a title="Search 'A2A'" href="/search?company=a2a&rf=13">A2A</a>
<a title="Search 'ABB'" href="/search?company=abb&rf=13">ABB</a>
<a title="Search 'Achmea'" href="/search?company=achmea&rf=13">Achmea</a>
etc... this continues for another ~6000 links
I have tried to use the following line to grab all the links, but this has not worked:
remDr$findElement(using="link text", value="href")
I'd be very grateful if someone could show me how to grab all the links, including the company names, such as 'A2A', 'ABB', 'Achmea', etc.
Regards,
mr_bungles

I suggest you use 'rvest' and 'tidyverse' along with RSelenium.
library(tidyverse)
library(rvest)
url <- 'add your url here'
pg <- read_html(url)
tbl <- tibble(
text = pg %>% html_nodes('add css selector here') %>% html_text()
link = pg %>% html_nodes('add css selector here') %>% html_attr('href')
)

Related

How to access Another Module's Content and Presentation Items after 2sxc v10.20+

Here is code that worked up through 2sxc 10.9.1. Though I am able to get the CmsBlock for the TabID, ModuleID and get that to .Render(), I need more. Here is the old code. Not sure it makes any difference, but this View is using the normal Link content-type and is running in an older version of the Content App (appx 3.03=ish). 2sxc has been upgraded and is now 11.22.0 LTS.
I have removed unnecessary stuff, so I doubt this runs as is...
#using ToSic.Razor.Blade
#using ToSic.SexyContent.Environment.Dnn7
#{
var Helpers = CreateInstance("_Helpers.cshtml");
// Display the items from the Manage Links module, we go in 'sideways'
// this gives us just the Content items with their Presentations settings, etc.
var sxci = Factory.SxcInstanceForModule(3360, 606); // ModuleID of Manage Links
var dyn = Factory.CodingHelpers(sxci);
var allLinks = dyn.AsDynamic(dyn.Data["Default"]);
}
#* other stuff *#
<div class="row co-documents justify-content-center align-items-center">
#foreach (var linkItem in allLinks) {
var linkInfo = Helpers.LinkInfos(linkItem.Link, linkItem.Window, linkItem.Icon);
string iconStyle = linkItem.IconStyle ?? "fas";
int linkColumns = (int)linkItem.Presentation.Columns;
string linkIconAlign = linkItem.Presentation.IconAlign;
string linkIconBGColor = linkItem.Presentation.IconBGColor;
#* other stuff *#
}
</div>
So the easy thing to figure out was how to get the module as a CmsBlock which I can Render() as is (below), but what I need to do instead is get proper access to the List of Content Items AND their Presentation data (like above, allLinks).
ToSic.Sxc.Dnn.Factory.CmsBlock(606, 3360).Render();
What am I missing? How can I get access to the other module's data like I was doing before? In this case, I do this in 3 different places on the website. So to outline this in English, I have a module that the client manages a few special links that get displayed in MegaMenus, other special nav, and directly on a couple of pages. In each place they render differently. In their "home" module, where they get edited, they just look boring like this:
I realize its something like this:
var allLinks = something1.AsList(something2.Data["Default"]);
I understand that something2 is an app instance, but how do I create it in the context of the other module?
And what is something1 nowadays? And how do instantiate it? Looks like its a new ToSic.Sxc.Code.DynamicCode() but I can't figure out how to construct that in a way that I can use or doesn't just throw errors.
Thanks in advance for any insight!!
Okay, it took a little testing, trial and error. And also I missed that DynamicCode() was a Method of the Factory class. In retrospect it does seem easy now.
So first you get the BlockBuilder
var block = Factory.CmsBlock(606, 3360);
Then you get the DynamicCode instance (Code.DnnDynamicCodeRoot) from that
var dc = Factory.DynamicCode(block);
And then things are normal
var allLinks = AsList(dc.Data["Default"]);
The rest of the code works like it did before; I can foreach through the links with Header (renamed from ListContent) and Presentation (now Content.Presentation) working just as expected.
The above answer works fine if you are inside the C# Razor template of the 2sxc View. But what if you are outside, for example in a Razor template for a DDR Menu?
Same two steps as above (get the block and the dc), but then you do NOT have access to AsList() or the App. Thankfully, you already have DynamicCode, so you could just get all the records in the Bibliography content-type like this:
<ul>
var items = dc.AsList(dc.App.Data["Bibliography"]);
foreach (var item in items)
{
<li>#item.EntityTitle</li>
}
</ul>
So once you've got your dc you've got access to all the usual 2sxc toys.

how can I load a joomla module as a link?

this is my problem...
I have some of images and links that I want to load different joomla modules when user click on them.
mean each hyperlink can load another module|position
thanks all
In case that you just want to call a module's content from a url the following answer will help you.
If you just want to show / hide a module in the same page you could use something similar to my previous answer: Joomla 3 Show different modules on same position depending on toggler
Joomla provides the functionality to call a specific file of the active template by adding the tmpl=FILENAME key/value to the url's query string.
All built-in templates have a component.php file if user wants to load the template with the component only. You could check the following link for more details: Adding print pop-up functionality to a component.
You could do something similar to only show the modules that you want to load.
You could copy the component.php to a new file (I have used custom.php) and added the following php code in the <body> ... </body> part.
<?php
$jinput = JFactory::getApplication()->input;
$selectedPosition = $jinput->getString("position", "");
$selectedModule = $jinput->getString("module", "");
$selectedModuleTitle = $jinput->getString("title");
if($selectedPosition !== "") {
$modules = JModuleHelper::getModules($selectedPosition);
foreach ($modules as $module) {
echo JModuleHelper::renderModule($module);
}
} elseif ($selectedModule !== "") {
$module = JModuleHelper::getModule($selectedModule, $selectedModuleTitle);
echo JModuleHelper::renderModule($module);
}
?>
So with a similar way as loadposition / loadmodule works you could call the new template file using:
index.php?tmpl=custom&position=MODULE_POSITION
or
index.php?tmpl=custom&module=MODULE_TYPE
or
index.php?tmpl=custom&module=MODULE_TYPE&title=MODULE_TITLE
Optionally if you want to load the module with a specific style, you could pass it to the second paramter of the renderModule method like:
echo JModuleHelper::renderModule($module, array("style" => "xhtml"));
Hope this helps

Links in strings - Typescript

I'm working at a project written in Ionic/Angular/Typescript. In the .html file, I have
< p> {{stringVar}} </p>
In the .ts file,I have
this.stringVar= "Visit http://www.google.com.
Visit http://www.stackoverflow.com."
I have 2 questions:
1) I want the 2 sentences in the string to be displayed in html on different lines. What should I do in order to achieve this: add \n or < br> or something like this?
2) I want the 2 links in the string to appear as links in html,too. That is,when the user clicks on them,he will be taken to those sites.
Thanks in advance!
1) To appear in different lines, you must put each one inside their own <p> tag, like this:
<p>first line</p>
<p>second line</p>
2) To appear as clickable links, you need to put in <a> tags, with url in href attribute, like this:
<p>click here to visit google.</p>
It would be better if you could change the structure of your data, to something like this:
<p ng-repeat="url in urlList">Visit {{url}}</p>
this.urlList = [
"http://www.google.com",
"http://www.stackoverflow.com"
];
or even better:
<p ng-repeat="site in siteList">Visit {{site.name}}</p>
this.siteList= [
{ name: "Google", url: "http://www.google.com" },
{ name: "StackOverflow", url: "http://www.stackoverflow.com" }
];
The best approach to go with a 'list', rather than a stringVar
this.linkList = [
"http://www.google.com",
"http://www.stackoverflow.com"
];
1) I would suggest to have <p></p> instead of <br/> in between.
2) The following is a working sample with Angular2
<p *ngFor="let link of linkList">Visit {{link}}</p>
Check the working sample here : https://embed.plnkr.co/Om3CXpT9xN07YCz2aHQr/
Both Question has one answer you Basically want to Interpolate string with html in the angular, although i am not expert in angular1.x but yes there is one service used for the same called as
$interpolate(templateString)(you_.ts/js_code);
by using this you can show your string as it as on the webpage event using html in you javascript file too. you just have to pass the html in your string ans display it in the webpage
for example lets assume your use case you simple have to add this like :-
this.stringVar= "Visit <a href='http://www.google.com'>Google</a> Here and<br> Visit <a href='http://www.stackoverflow.com'>Stackoverflow</a> Here."
and than convert this using interpolate like this
$scope.your_string = $interpolate(templateString)(stringVar);
Working Example for the same

Use variable inside Hugo content

I'm trying to use a variable within the content of a Hugo statically generated site. For example, the content looks like the following:
Go to your site's url ({{ .Site.BaseURL }})
Enter your credentials
.....(blah blah blah)
When this gets rendered, the {{ .... }} part doesn't get processed...it stays the same as I put above. I've tried it with a $ in front as well. Variables within templates seem to work just fine. Do I need to create a shortcode to use within content pages?
So it looks like a shortcode is the way to do this. For what it's worth, I changed the document to look like the following:
Go to your site's url ({{< siteurl >}})
In layouts/shortcodes, I created the file siteurl.html. It looks like the following:
{{ .Page.Site.BaseURL }}
I needed to add .Page in there to get access to the Site variables. See this Issue Report for more details.
In Hugo, When you want to use a variable in markdown (.md) file then you need to create a shortcode for that first.
You can follow these steps:-
create shortcode
layouts/shortcodes/siteurl.html
{{ .Page.Site.BaseURL }}
usage
content/post/myblogpost.md
---
# front-matter
---
1. Go to your site's url ({{< siteurl >}})
2. Enter your credentials
3. .....(blah blah blah)
result
post/myblogpost.html
1. Go to your site's url (https://codingnconcepts.com)
2. Enter your credentials
3. .....(blah blah blah)
Source: https://codingnconcepts.com/hugo/custom-shortcode-hugo/
I had the same problem, and this post helped me.
I wanted to display a site param in my site content, and discovered you cannot use regular templating inside content files.
In the end I created a shortcode to load the requested site param. Who knows this information might help someone.
/config.yml
params:
appName: My app
/content/about.html
<p>My app's name is {{< param "appName" >}}</p>
/layouts/shortcodes/param.html
{{/* Usage: {{< param "siteParamName" }} */}}
{{ index .Site.Params (.Get 0) }}
Result
<p>My app's name is My app</p>
This is an attempt to slightly improve #minitauros answer with a simplistic example to lookup a (site) parameter sub-key (aka walk the YAML tree, infer an element, etc.).
I would like Hugo to have a JSONPath or jq syntax and, obviously, this example is far from competing with either solutions.
config.yml
params:
mode: one
support:
mailing: info#example.net
layouts/shortcodes/param.html
{{ $v := .Site.Params }}
{{ range (split (.Get 0) ".") }}{{ $v = index $v (.) }}{{ end }}
{{ $v }}
content/_index.md
We are in mode {{< param "mode" >}}.
In case of turbulence, [reach the support](mailto:{{< param "support.mailing" >}}) for help.

XPages: File Download control link url

I have a file download control that lists attachments from some documents in my database.
I want to display an icon next to each row and make it a link to the attachment of the row.
If not sure how to do it for each row, let's assume that i have only 1 row. How can i get the link of the attachment so as to declare it as href in a link control?
As i already mentioned in my Comment if you are using a <xp:fileDownload> you can add a Icon if you set displayType="true" and because you didnt add code to your question i guess your code could look something like this:
//..your code
<xp:panel id="row">
<xp:this.data>
<xp:dominoDocument
var="document1"
action="openDocument"
documentId="#{javascript://example... viewEntry.getDocument().getUniversalId()}">
</xp:dominoDocument>
</xp:this.data>
<xp:fileDownload
rows="30"
id="fileDownload1"
displayLastModified="false"
value="#{document1.Body}"
displayType="true">
</xp:fileDownload>
</xp:panel>
//..your code
or if you dont use a <xp:fileDownload> and maby just Display rows with the attachment Name you could use something like this:
//... your code
<xp:panel id="row">
<xp:repeat
id="repeat1"
rows="30"
value="#{javascript:#AttachmentNames()}"
indexVar="attachmentIndex"
var="attachment">
<xp:link
escape="true"
text="#{javascript:attachment;}"
id="link1"
target="_blank">
<xp:this.value><![CDATA[#{javascript:
var url = facesContext.getExternalContext().getRequest().getContextPath() + "/0/" +
/*in my case: viewEntry.getDocument().getUniversalID()*/
+ "/$File/"+ AttachmentName;
return url;}]]></xp:this.value>
<xp:image id="image1">
<xp:this.url><![CDATA[#{javascript://
var pdfImage = 'pdf.gif';
if(attachment.indexOf("pdf")> 0)
return pdfImage;
}]]></xp:this.url>
</xp:image> 
</xp:link>
<br></br>
</xp:repeat>
</xp:panel>//...your code
The <xp:repeat> inside your row will create a link for each attachment inside of your document you can remove it if you only have one attachment per document.

Resources