A/B Test solutions for a locked source code environment - server side decides which user gets which variation - google-website-optimizer

I've been previously using tools such as Google Website Optimizer to run multi-variation or A/B tests. However right now I am looking for a solution that works for a larger site (400-500 000 unique visitor per month) with a very locked down source code environment. Basically:
The site is balanced over several servers
All code that is to be released on any of those servers must go via version control, unit testing and acceptans testing. All releases must be signed by develop, sys-admin and test executive.
This means that I am not allowed/it's hard to add "new code" (even if it's tested and verified) via Google Website Optimizer or any other of the GUI-paste-your-new-variation-here type of solutions.
We can however on server side decide which users gets which variation. Basically we can push the new version on X of the servers making 10-30% of the users view it for their entire session. The question is: Which tools do we use to measure "success" (i.e improved conversion rate). My idea so far has been:
Tag the new version in Google Analytics using a session variable (and then make reports based on segment) (similar to what is described on http://searchengineland.com/how-to-analyze-ab-tests-using-google-analytics-67404 )
Use Optimizely which has API support:
window.optimizely = window.optimizely || [];
window.optimizely.push(['bucketUser', EXPERIMENT_ID, VARIATION_ID])
What solutions have you tried for locked-down environments? Am I missing some obvious solution?
The site is in .NET/Episerver on IIS.
Regards,
Niklas

You could use the AB-testing capability built into EPiServer CMO.

We ended up going with Google Analytics and adding a session variable such as "abtest" with value "variation-4" and publishing it on certain nodes. It worked fairly well, with some limitations, namely that google analytics funnels doesn't have segment support.

We did something similar and we have found Google Analytics documentation confusing. In the end the following code (made by server) got the work done for us:
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-xxxx-xxxx', {
'custom_map': {'dimension1': 'abTestDesign'}
});
gtag('event', 'abTestDesign_dimension', {'abTestDesign': 0, 'non_interaction': true});
</script>
This code is generated by the server, where the last JS line is either that one of
gtag('event', 'abTestDesign_dimension', {'abTestDesign': 1, 'non_interaction': true});
Seems to be working fairly well on Numbeo.com

Related

Azure Form Recognizer - Copy model from QA to PROD

Our team has built more than 1000+ models in development environment and tested the output. We moved the models from QA to Production using StartModelCopyTo method using Form Recognizer client SDK. During each copy model, code is written such a way that if PercentageCompleted is 100%, then move to next model. All 1000+ models copied to production service. Now the problem is, when we use GetCustomModels method to list all models, all models comes as null, but if I use model Id, it returns all details. Has anyone faced this issue? Business team considers this as an issue and not ready to sign off. We are facing other issues with the Form Recognizer service too.
This could be related to the SDK/REST API version you are using for the get operation. Can you validate that you are using the API version and SDK that corresponds to the V2.1 or v3 API based on which version the mode was trained with?
Direct message if you are still having trouble.
Microsoft support came back and said that copy model method has threshold of 1 per minute. Now we created a delay of 1 per minute and it works.

iOS11 location update after getting killed by user

I'm implementing a non VOIP app in iOS 11. As per the nature of my App, I've implemented the startMonitoringSignificantLocationChanges(). All the implementation went well and it is working now.
while terminating the app I'm calling startOrReStartSignificantLocationUpdate()
and I'm restarting significantlocationupdate in didFinishLaunchingWithOptions
also I'm updating location in server in didUpdateLocations event
Location updates is reflected on my server when the App is in foreground or background. It is now reflects even if user kill the App manually. So it works with SLC but
the problem is;
Updates seems to get stopped after a period of time say, 3 - 4 hours. It happens if the user kills the App manually. As per the nature of my App, I need to get it re-initiated up on location change and get the location update to my server.
Is there any category of App which can get this location updates forever in iOS11? I'm thinking about how Moves App in iOS works. Not sure it works perfect with iOS11 though
If you only use “While Using” location access, as far as I understand nothing changes for you. If you use the “Always” access though, and you didn’t provide a “While Using” fallback as you should have… well, you will need to do that now
As Brad Jensen explains in the WWDC talk I mentioned, Apple has decided that forcing the user to give the app “Always” location access is a bad user experience:
Now, a second reason we think that many developers choose to require
Always authorization is they’re simply trying to give their users the
best possible experience (…) but this makes for a very poor user
experience for the users that don’t wish to grant the app Always
authorization. They are forced to choose between granting the app
Always, which is more than they would like in this example, or
granting it Never, which means they don’t get to benefit from any of
the app’s location-based features. And their final option is to grant
it Always and then revoke that authorization after they’re done using
the app. In any case, the user is not having a great time with this
app.
So, when you build your app using the iOS 11 SDK, you are now required to provide an NSLocationWhenInUseUsageDescription key in all cases (if you use location at all). For “Always” access, you can additionally provide a second key, which is now called NSLocationAlwaysAndWhenInUseUsageDescription. If you only provide NSLocationAlwaysAndWhenInUseUsageDescription but not NSLocationWhenInUseUsageDescription, asking for “Always” access will not work:
This app has attempted to access privacy-sensitive data without a
usage description. The app's Info.plist must contain both
NSLocationAlwaysAndWhenInUseUsageDescription and
NSLocationWhenInUseUsageDescription keys with string values explaining
to the user how the app uses this data
Effect on existing apps
Most SDK changes usually only take effect once you recompile your app with the latest version of the SDK & Xcode. This one is different: even if you don’t release any updates this summer, this change will affect you.
Here’s what the user sees when they run existing apps from the App Store in iOS 11 beta and they ask for location access:
The first one is from Foursquare, which has provided both access level options for a long time. In this case, iOS 11 lets you immediately choose “While Using” or “Always”, and since the description texts have not yet been updated with the new behavior in mind, it shows both texts at the same time.
The second one is from Topo Maps, an app that lets you browse hiking maps and find yourself on the map while you’re on the trail. You can imagine it has no reason to track you in the background, but still, the developer has not provided a “While Using” option. iOS 11 now lets you choose that option anyway, but since it might possibly cause some unexpected behavior in the app, it shows a warning that some parts of the app might not work.
If your app has only provided an “Always” key so far, I’d recommend that prepare a transitional update that adds the NSLocationWhenInUseUsageDescription key and makes sure the app works properly in this mode. That way, the user will get a popup like the one in Foursquare, which at least looks less scary.

google apps from app engine

I want to produce a Google Apps document based on a (Google doc) template stored on the users Google Drive and some XML data held by a servlet running on Google App Engine.
Preferably I want to run as much as possible on the GAE. Is it possible to run Apps Service APIs on GAE or download/manipulate Google doc on GAE? I have not been able to find anything suitable
One alternative is obviously to implement the merge functionality using an Apps Script transferring the XML as parameters and initiate the script through http from GAE, but it just seem somewhat awkward in comparison.
EDIT:
Specifically I am looking for the replaceText script functionality, as shown in the Apps script snippet below, to be implemented in GAE. Remaining code is supported through Drive/Mail API, I guess..
// Get document template, copy it as a new temp doc, and save the Doc’s id
var copyId = DocsList.getFileById(providedTemplateId)
.makeCopy('My-title')
.getId();
var copyDoc = DocumentApp.openById(copyId);
var copyBody = copyDoc.getActiveSection();
// Replace place holder keys,
copyBody.replaceText("CustomerAddressee", fullName);
var todaysDate = Utilities.formatDate(new Date(), "GMT+2", "dd/MM-yyyy");
copyBody.replaceText("DateToday", todaysDate);
// Save and close the temporary document
copyDoc.saveAndClose();
// Convert temporary document to PDF by using the getAs blob conversion
var pdf = DocsList.getFileById(copyId).getAs("application/pdf");
// Attach PDF and send the email
MailApp.sendEmail({
to: email_address,
subject: "Proposal",
htmlBody: "Hi,<br><br>Here is my file :)<br>Enjoy!<br><br>Regards Tony",
attachments: pdf});
As you already found out, apps script is currently the only one that can access an api to modify google docs. All other ways cannot do it unless you export to another format (like pdf or .doc) then use libraries that can modify those, then reupload the new file asking to convert to a google doc native format, which in some cases would loose some format/comments/named ranges and other google doc features. So like you said, if you must use the google docs api you must call apps script (as a content service). Also note that the sample apps script code you show is old and uses the deptecated docsList so you need to port it to the Drive api.
Apps script pretty much piggy backs on top of the standard published Google APIs. Increasingly the behaviours are becoming more familiar.
Obviously apps script is js based and gae not. All the APIs apart from those related to script running are available in the standard gae client runtimes.
No code to check here so I'm afraid generic answer is all I have.
I see now it can be solved by using the Google Drive API to export (download) the Google Apps Doc file as PDF (or other formats) to GAE, and do simple replace-text editing using e.g. the iText library

Google Cloud Endpoint : Request params not delivered to Cloud Endpoints when using Google APIs Client Library for JavaScript

Suddenly my gapi client stopped sending request params to endpoint.
This is how my code looks like
Load the gapi JS
https://apis.google.com/js/client.js?onload=initGoogleApis
in initGoogleApis
function initGoogleApis() {
var ROOT = HOST + "/_ah/api";
gapi.client.load("userendpoint", "v1", function() {
userendpoint = gapi.client.userendpoint;
}, ROOT); }
Now when I query userendpoint.<some function>, then it is not passing the request params to endpoint
NOTE: it was working fine till today morning.
Anyone else facing the same issue? (this might be due to some update in the gapi library)
This issue has been resolved as of yesterday 2014-09-23 08:00 (US Pacific Time).
Details about this issue can be found in the Google App Engine Downtime Notify Group
However 'Google APIs Client Library for JavaScript' is still in Beta and breaking changes have been rolled out more than once. Clound Endpoints themselves are out of beta and can be used for production use.
Now, to properly answer this questions:
The simple advice here is: Don't use beta products for production applications.
To avoid problems with Google APIs Client Library for JavaScript, just don't use it. You can write your own REST API client that will not be affected by changes to the JavaScript library from Google. I have done this for testing purposes a couple of times and it is not hard, just a lot of work depending on how many endpoints you have and how complex they are.
We have the same problem on two projects.
I think that Google has deoployed a new version of the "https://apis.google.com/js/client.js" and it dosen't works as expected...
We need to open a ticket to Google support. If I have any news I will report them to you.
Google reports (https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/t9GElAJwj8U):
We are currently experiencing an issue with Google Cloud Endpoints where the GAPI Javascript client is unable to pass request parameters. For everyone who is affected, we apologize for any inconvenience you may be experiencing. We will provide an update by Tuesday, 2014-09-23 05:00 (all times are in US/Pacific) with current details, and if available an estimated time for resolution.
Update:
We have fixed the issue affecting Google Cloud Endpoints JavaScript client and are gradually rolling-out a fixed version. We estimate full resolution of the issue by 06:30 US/Pacific Pacific. We will provide an update by 06:00 AM.
Update:
Now it works for me.
Marco

HTML5 database storage (SQL lite) - few questions

Hy there,
I can't find enough beginner resources on the web about HTML5 database storage usage examples (CRUD)
I'm opening(creating) my DB like this:
var db;
$(document).ready(function()
{
try
{
if (!window.openDatabase) {
alert('Not Supported -> Please try with a WebKit Browser');
} else {
var shortName = 'mydatab';
var version = '1.0';
var displayName = 'User Settings Database';
var maxSize = 3072*1024; // = 3MB in bytes 65536
db = openDatabase(shortName, version, displayName, maxSize);
}
}
catch(e)
{
if (e == 2) {
alert("Invalid database version.");
} else {
alert("Unknown error "+e+".");
}return;
}
});
QUESTION 1: How many databases can i create and use on one domain?
QUESTION 2. How to delete (drop) a database. -> i have not figured this out yet.
To create sql queries you use transaction:
function nullDataHandler(transaction, results) { }
function createTables(db)
{
db.transaction(function (transaction)
{
//first query causes the transaction to (intentionally) fail if the table exists.
transaction.executeSql('CREATE TABLE people(id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL DEFAULT "John Doe", shirt TEXT NOT NULL DEFAULT "Purple");', [], nullDataHandler, errorHandler);
});
}
QUESTION 3: How so is the above transaciton failed if a table exists? Is the nullDataHandler involved to do this? Where on the web is there documentation explaining the executeSql API? Arguments?
thx
The spec you're looking for is Web SQL Database. A quick reading suggests:
There is no limit, although once your databases increase beyond a certain size (5MB seems to be the default), the browser will prompt the user to allow for more space.
There is no way, in the current spec, to delete databases.
The executeSql() function takes an optional error callback argument.
HTML5 Doctor also has a good introduction.
Going forward, though, I'd recommend looking at Indexed DB. Web SQL has essentially been abandoned since there is no standard for SQL / SQLite. Even Microsoft has endorsed Indexed DB. See Consensus emerges for key Web app standard.
CREATE TABLE IF NOT EXISTS table_name
will create a table table_name only if if does not exist.
I found the following WebSQL tutorials helpful for basic CRUD operations, as they contained examples, and explained what the code was doing:
A Simple TODO list using HTML5 WebDatabases
HTML5 Web SQL Database
And the following links for SequelSphere (an HTML5 JavaScript SQL Relational Database Alternative to WebSQL that works in all browsers, storing data in LocalStorage and IndexedDB):
SequelSphere basic Usage instructions
API Documentation
Full Index of Guides and Helper Documentation
Using PersistenceJS there is a persistence.reset API which will wipe the database clean.
PersistenceJS Site
For developing / testing purposes, you can view content and delete webSQL, IndexedDB, cookies, etc by searching for your domain name at this URL in Chrome:
chrome://settings/cookies
There, you can delete all the storage for a domain or just certain local storage entities. Yes, the URL implies just 'cookies', but the interface at this URL includes all types of offline storage.
It would be great I think if the Chrome developer tools interface had the ability to right-click and delete a data storage entity in the Resources tab along with inspecting the content. But for now, all I know of is the settings/cookies URL.
It is supported on iOS safari,chrome and some latest version of opera....it's not yet adopted by IE and Firefox that's it......what more one can ask than local db on browser which has relational db system...so u can query it easily and handle complex data....which is very tougher in key vale based systems..
I remember reading it even supports upto one gb.i am not sure....
Note:
1)I'd like to mention one point there is a IDE called Dashcode which let's u build web apps that looks like iOS native.even there also web SQL is used.
2)actually web SQL is a implementation of SQLite on browsers.
3)SQLite is most prefered in both iOS and android as db for native code..
The drawbacks of SQLite:
The Lack of concurrency support but which is not a problem in browser as it's gonna be used by single user at a time..this is a case also in mobile.
Conclusions:
Web Sql is abandoned by w3 that's a sad thing so we've to explore other options.

Resources