How to curl/needle/request google.flight correctly?

How to curl/needle/request google.flight correctly? - request

I want to parse the page (https://www.google.com/flights/#search;f=JFK;t=SFO;d=2014-12-22;r=2014-12-30) on my backend using needleJs(https://github.com/tomas/needle) and Cheerio(https://github.com/cheeriojs/cheerio).
When I tried to request the google.flight page by the url, what I got is
> <!DOCTYPE html> <html><head><meta http-equiv="content-type"
> content="text/html; charset=UTF-8"><meta name="flights::gwt:property"
> content="baseUrl=/flights/static/"><title>Flights - Google
> Search</title> <meta name="description" content="Choose your flight
> from a simple list of results, explore destinations on a map, and find
> travel dates with the lowest fare with Flight Search."><script
> language="javascript" type="text/javascript"> var __JS_ILT__ = new
> Date(); </script> <style
> type="text/css">#gbar,#guser{font-size:13px;padding-top:1px
> !important;}#gbar{height:22px}#guser{padding-bottom:7px
> !important;text-align:right}.gbh,.gbd{border-top:1px solid
> #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}#media
> all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline
> !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27
> !important}.gbf .gb4{color:#900 !important}</style><script
> language="javascript" type="text/javascript"> var __JS_INI__ =
> "[,[,[,[1275,1105,1266,1174,1144,1203,15,16,10]],[,\42USD\42,\42$\42,\42\\u00a4#,##0\42,\42\\u00a4#,##0.00\42,\42#,##0\42,1],[,2,1609.344,\42#,##0\42,\42#,##0.0\42],[,2,2.54,\42#,##0\42],\42google-travel\42,\42typeId:72275\42,\42ADS25WNJCodY_Q5E8phMiFpRv5Mjahv3VPknsbbucoHBj4CEkSDnkC31pFues4Idsta-tlsZHS3HRUa2nBAqRARRiSySmvbRZymx5Q\42,\42.com\42,\42en\42,\42US\42,1,,,,,,\42https://accounts.google.com/ServiceLogin?continue\\u003d_CONTINUE_\42],[,\42BOS\42,\42Boston\42,42.3644444,-71.005278,[\42BOS\42,\42PSM\42],[\42BOS\42],\42United
> States\42,\42MA\42,\42US\42,\42Massachusetts\42,,42.3584308,-71.0597732],[[,\42BOS\42,\42Boston
> Logan International\42,\42Boston\42,\42BOS\42,\42United
> States\42,42.3644444,-71.005278,\42MA\42,\42US\42]],[[,\42ONEWORLD\42,\42Oneworld\42],[,\42SKYTEAM\42,\42SkyTeam\42],[,\42STAR_ALLIANCE\42,\42Star
> Alliance\42]],,[[,\42_fli\42,\42Flights\42],[,\42_mor\42,\42More\42],[,\42_web\42,\42Web\42],[,\42#map\42,\42Maps\42],[,\42nws\42,\42News\42],[,\42shop\42,\42Shopping\42],[,\42isch\42,\42Images\42],[,\42vid\42,\42Videos\42],[,\42bks\42,\42Books\42],[,\42app\42,\42Apps\42]],1,,\42US
> Dollar\42,0]"; </script> <script>(function(){var
> gs=document.createElement('script');var wmm=window.matchMedia;var
> hires=!!wmm && !wmm('(-webkit-device-pixel-ratio:1.0)').matches &&
> !wmm('(-moz-device-pixel-ratio:1.0)').matches;gs.src=!hires?'static/D4C5482E0AEB161B751BBFC9C57F4C0D.cache.js':'static/7521F8CC51FAFDF1C1478664FF1CF4BD.cache.js';gs.type='text/javascript';gs.async=true;document.getElementsByTagName('head')[0].appendChild(gs);})();</script><script
> language="javascript" type="text/javascript"></script> <link
> rel="shortcut icon" href="/favicon.ico"/><meta name="google"
> value="notranslate"></head><body> <script language="javascript"
> type="text/javascript">(function() { var script =
> document.createElement('script'); script.type = 'text/javascript';
> script.src = 'https://ssl.gstatic.com/feedback/api.js';
> document.body.appendChild(script);})();</script> <div id=gbar><nobr><a
> class=gb1 href="https://www.google.com/search">Search</a> <a class=gb1
> href="https://www.google.com/search?hl=en&tbm=isch&source=og">Images</a>
> <a class=gb1 href="https://maps.google.com/maps?hl=en">Maps</a> <a
> class=gb1 href="https://play.google.com/?hl=en">Play</a> <a class=gb1
> href="https://www.youtube.com/results">YouTube</a> <a class=gb1
> href="https://news.google.com/nwshp?hl=en">News</a> <a class=gb1
> href="https://mail.google.com/mail/">Gmail</a> <a class=gb1
> href="https://drive.google.com/">Drive</a> <a class=gb1
> style="text-decoration:none"
> href="http://www.google.com/intl/en/options/"><u>More</u>
> »</a></nobr></div><div class=gbh style=left:0></div><div
> class=gbh style=right:0></div><div id="_BrowserWarning_"
> style="border:1px solid
> #FFE475;background-color:#FEF7CB;padding:4px;text-align: center;color: #222;">Google Flight Search has not been optimized for your browser. For best results, please try Chrome, Firefox 3.5+, Internet Explorer
> 8+, Safari 4+.<a href="#" style="color: #222;"
> onclick="document.getElementById('_BrowserWarning_').style.display='none';
> return false;"> Close</a></div><div id="root"></div></body></html>
Which is not the page I want.
For performance purpose, I guess the reason is that Google tries to dump a very small size of html when I request. And there are a few necessary javascript codes in the html which will be executed after the DOM loaded. And then those script will fetch the fares and render them. Correct me if I am wrong.
So what can I do in order to parse the right page by the url for the Google flight site??
Thanks.

You could use https://developers.google.com/qpx-express/faq#q1 instead. There is a free daily quota.

Related

Disable field.hide in angular formly

The default hide directive with angular-formly is ng-if which can be configured via e.g. the formlyConfigProvider.
Currently all my fields should always be shown and I don't want to have unneccesary ng-if="!field.hide" checks rendered that can inpact the performance.
How can I tell formly not to use this check per field/form or globally?

ng-if add and remove elements from the DOM, when you want to show and hide large number of elements it can be slow, insted you can use ng-show.
ng-show will only change the visibility of the element.
<html lang="en" itemscope="" itemtype="http://schema.org/Article">
<head>
<script>
var oModelesDep = [];
</script>
<!-- Angular Material requires Angular.js Libraries -->
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.5/angular.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.5/angular-animate.min.js"></script>
<script src="https://s3.amazonaws.com/gowpres/resources/js/utils/jquery-1.10.2.js"></script>
<!-- Angular Material Library -->
<script src="https://www.weldpad.com/starterkit.js?containerId=60515"></script>
<script data-meta="61021" src="https://www.weldpad.com/sogettopanswerers.html?containerId=61021"></script>
</head>
<body ng-controller="AppCtrl" ng-app="MyApp">
<h4 ng-init="showchat = true">Your - Starter Kit</h4>
<button ng-click="showchat = false">hide</button>
<button ng-click="showchat = true">show</button>
{{showchat}}
<sogettopanswerers ng-show="showchat" tag="html">
<div ng-repeat="qdata in listqdata.items track by $index" style="background-color: white;">
<div class="well" style="overflow: auto;">
<h2>
<a href="{{qdata.link}}" class="question-hyperlink">
{{qdata.title}}
</a>
<small>{{qdata.view_count}} Views</small></h2>
<contentashtml ng-init="load()" content="qdata.body">
</contentashtml>
<div style="padding:15px;display: inline-block;vertical-align: top;">
<p>Name: {{qdata.owner.display_name}}</p>
<a href="{{qdata.owner.link}}">
<img ng-src="{{qdata.owner.profile_image}}" alt="Description"/>
</a>
</div>
<div style="display: inline-block;">
<p>Created: <span am-time-ago="qdata.creation_date * 1000"></span></p>
<p>
Last Update:<span am-time-ago="qdata.last_activity_date * 1000"></span>
</p>
<p>
Answered:{{qdata.is_answered}}
</p>
</div>
<p>
Answers:{{qdata.answer_count}}
</p>
</div>
</div>
</sogettopanswerers>
</body>
</html>
Look at the line:
<sogettopanswerers ng-show="showchat" tag="html">
and see how fast the response is.

you set hide-directive="ng-show" in the formly-form
<formly-form hide-directive="ng-show"></formly-form>
"hide-directive
Allows you to control the directive used to hide fields. Common value for this might be ng-show. It will be passed !field.hide. You can also specify this on a global level using formlyConfig.extras.defaultHideDirective = 'ng-show'"
http://docs.angular-formly.com/docs/formly-form
So you can either set it as I instructed or you can choose to edit it in the config on startup for all fields

How to dismiss a tutorial popover by clicking outside in an Angular JS site?

I'm testing an Angular site with Protractor and Jasmine framework.
How can i locate the mouse outside the tutorials popover that is displayed at the login time in order to hide this popover to test the home page?
Here the html code that contain the button of tutorial (previous; next; don't show again) :
</div></md-card-content> </md-card><!-- end ngIf: !expandChart --> </div> </div> </div></md-content> </div></div> <!-- Google Analytics: change UA-XXXXX-X to be your site's ID --> <!--<script>--> <!--!function(A,n,g,u,l,a,r){A.GoogleAnalyticsObject=l,A[l]=A[l]||function(){--> <!--(A[l].q=A[l].q||[]).push(arguments)},A[l].l=+new Date,a=n.createElement(g),--> <!--r=n.getElementsByTagName(g)[0],a.src=u,r.parentNode.insertBefore(a,r)--> <!--}(window,document,'script','https://www.google-analytics.com/analytics.js','ga');--> <!--ga('create', 'UA-XXXXX-X');--> <!--ga('send', 'pageview');--> <!--</script>--> <script src="scripts/vendor.js"></script> <script src="cordova.js"></script> <script src="scripts/scripts.js"></script> <script src="https://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha1.js"></script> <script src="https://crypto-js.googlecode.com/svn/tags/3.1.2/build/rollups/sha256.js"></script> <script src="https://crypto-js.googlecode.com/svn/tags/3.1.2/build/components/enc-base64-min.js"></script> <div class="introjs-overlay" style="top: 0;bottom: 0; left: 0;right: 0;position: fixed;opacity: 0.8;"></div><div class="introjs-helperLayer " style="width: 538px; height:366px; top:64px;left: 195px;"></div><div class="introjs-tooltipReferenceLayer" style="width: 538px; height:366px; top:64px;left: 195px;"><div class="introjs-tooltip" style="left: 546px;"><div class="introjs-tooltiptext">Watchlist view. Swipe the row in the grid to the left to show the delete action.</div><div class="introjs-bullets"><ul><li><a class="active" href="javascript:void(0);" data-stepnumber="1"> </a></li><li> </li><li> </li><li> </li><li> </li><li> </li><li> </li><li> </li></ul></div><div class="introjs-progress" style="display: none;"><div class="introjs-progressbar" style="width:12.5%;"></div></div><div class="introjs-arrow left" style="display: inherit;"></div><div class="introjs-tooltipbuttons"><a class="introjs-button introjs-skipbutton" href="javascript:void(0);">Don't show it again!</a>PreviousNext</div></div></div></body></html>

How i add text in iframe in e2e testing Protractor

how i add text in p tag?
<div id="cke_1_contents" class="cke_contents cke_reset" role="presentation" style="height: 350px;"><span id="cke_44" class="cke_voice_label">Press ALT 0 for help</span>
<iframe class="cke_wysiwyg_frame cke_reset" frameborder="0" src="" style="width: 100%; height: 100%;" title="Rich Text Editor, editor2" aria-describedby="cke_106" tabindex="0" allowtransparency="true">
<html lang="en" dir="ltr">
<head>
<body class="cke_editable cke_editable_themed cke_contents_ltr cke_show_borders" contenteditable="true" spellcheck="true">
<p>
<br type="_moz">
</p>
</body>
</html>
</iframe>
</div>
i use this code but the give error
var p = element(by.css('.cke_editable p'));
p.sendKeys('This is a peragraph tag');

You'll need to switch to the iframe's page context to do this:
browser.switchTo().frame($('.cke_wysiwyg_frame'))
var p = element(by.css('.cke_editable p'));
p.sendKeys('This is a peragraph tag');
// switch back to the "parent page" context
browser.switchTo().defaultContent()

How to bind filtered result to another action with Angular?

I am trying to develop action bar with search functionality.
Demo: http://plnkr.co/edit/lWoHw7Aiwacln4GAHfNm?p=preview
The action bar updates when selecting/deselecting files without any search query; however, I now want to bind the action bar to filtered result after entering a search query.
For example:
Select File A (Notice: Delete and Open actions are added)
Enter a in the search input. (Bug: Since there is only ONE item in the filtered result, I want the selectState to change to all. This will cause Select All to be replaced by Deselect All.)
Remove a form the search input. (Another Bug: Notice at this point selectState changes to all. Now Select All is replaced by Deselect All, but that is wrong because the filtered result now has a lot more elements and we haven't selected everything.)
<head>
<script data-require="angular.js#*" data-semver="1.3.0" src="//code.angularjs.org/1.3.0/angular.js"></script>
<script src="//cdnjs.cloudflare.com/ajax/libs/lodash.js/2.4.1/lodash.compat.min.js"></script>
<link href='http://fonts.googleapis.com/css?family=Open+Sans' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="style.css" />
<script src="script.js"></script>
</head>
<body ng-controller="FilesCtrl">
<h1>Search + Bar</h1>
<input ng-change="updateSelectState()" ng-model="search" type="text" placeholder="search" />
<div class="files_bar">
<ul>
<li ng-show="selectState == 'none' || selectState == 'one' || selectState == 'multiple'" ng-click="selectAll()">
<span>Select All</span>
</li>
<li ng-show="selectState == 'all'" ng-click="deselectAll()">
<span>Deselect All</span>
</li>
<li ng-show="selectState == 'one' || selectState == 'multiple' || selectState == 'all'" ng-click="confirmView()">
<span>Delete</span>
</li>
<li ng-show="selectState == 'one'">
<span>Open</span>
</li>
</ul>
</div>
<div class="files">
<div class="file" ng-repeat="file in filteredFiles = ( files | filter:{ name: search })">
<input type="checkbox" ng-checked="file.checked" ng-click="toggleFileSelect(file)">
{{file.name}}
</div>
<div ng-if="filteredFiles.length == 0">
<h4>No file found.</h4>
</div>
</div>
</div>
</body>
</html>

Convert append from in javascript to AngularJS

I created this simple game where a user is trying to guess a number. I want to use AngularJS frameworks but I am lacking on the basic concepts. For example the first part of my code is a a function that will be used to append the guess to a list of guesses. My issue here is I am not sure how to append to a scope in AngularJS. Here is that part of my original code
var numOfAttempts = 6;
var numOfGuesses = 0;
var numberToGuess = "Three";
function listWordsUsed(numberAttempted)
{
var userTrials = $('#userGuesses');
var divisor = $("<p id ='line'><div class='answers'>" + numberAttempted + "</div></p>");
divisor.hide().appendTo(userTrials).fadeIn(6000);
return;
} //End of function listWordsUsed(numberAttempted)
I know that I will have to make my variables scopes in order to be used by my aplication, like this
$scope.numOfAttempts = 6;
$scope.numOfGuesses = 0;
$scope.numberToGuess = "Three";
However I am not sure how to append to a scope variable in AngularjS. Any suggestions? Hints? By the way here is my HTML as I started setting it up.
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Guess the Number in AngularJS</title>
<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.0.7/angular.min.js" type="text/javascript"></script>
<script src="js/controllers/app.js" type="text/javascript"></script>
<script src="js/controllers/maincontroller.js" type="text/javascript"></script>
</head>
<body>
<div id='content' ng-app='numberApp' ng-controller='numberController'>
<p>
<header>
<h2 align="center">What number am I thinking of?</h2>
</header>
<h3>Enter the number:</h3>
<table>
<td><input type='text' ng-model='guestGuess' />
<p>This is: {{guestGuess}}</p>
<p>This is: {{magicWord}}</p>
</td>
<td id="guessBox"><input type="submit" id="guessButton" value="Guess" /></td>
</table>
</p>
<p>
<h3>Your guesses so far are: </h3>
<p style="text-align:center;" id="numberGuesses"></p>
<p id="guessesUsed">You have guessed: <span id="userAttempts"></span> times out six(6) chances.</p>
<p id="guessesLeft">You have <span id="attemptsLeft"></span> guesses left.</p>
</p>
</div>
</div>
</body>

Here's a Plunkr
Basically you just build an array of the guesses you list it with ng-repeat. The rest is just count with .length(), and then do basic math right in the interpolated HTML.
<div ng-controller="numberController">
<h3>Enter the number:</h3>
<input type="text" ng-model="guess" />
<p>Guess is: {{guess}}</p>
<button ng-click="addGuess()">Guess</button>
<h3>Your guesses so far are: <span>{{guessed}}</span></h3>
<ul>
<li ng-repeat="g in guesses">
<p>{{g}}</p>
</li>
</ul>
<p>You have guessed: <b>{{guessed}}</b> times out six({{allowed}}) chances.</p>
<p>You have <b>{{allowed - guessed}}</b> guesses left.</p>
</div>