I know lucene, just started to learn how to use solr. In the simple example, the way to add document is to used the example ../update -jar post.jar to add document, the question is without writing my own add document in java, using the same way (... post.jar), is there a way to add additional fields not in the document? For example, say my schema include name, age, id fields, but the document has no 'id' field but I want the id and its value to be included, of course I know what id and value I want but how do I include it?
Thanks in advanced!
I don't believe you can mix the two. You can use post.jar to add documents using arguments passed in on the commandline, a file, stdin or a simple crawl from a web page but there is no way to combine them. In the source code for post.jar you can see it's a series else if statements so they are mutually exclusive.
-Ddata args, stdin, files, web
Use args to pass arguments along the command line (such as a command
to delete a document). Use files to pass a filename or regex pattern
indicating paths and filenames. Use stdin to use standard input. Use
web for a very simple web crawler (arguments for this would be the URL
to crawl).
https://cwiki.apache.org/confluence/display/solr/Simple+Post+Tool
/**
* After initialization, call execute to start the post job.
* This method delegates to the correct mode method.
*/
public void execute() {
final long startTime = System.currentTimeMillis();
if (DATA_MODE_FILES.equals(mode) && args.length > 0) {
doFilesMode();
} else if(DATA_MODE_ARGS.equals(mode) && args.length > 0) {
doArgsMode();
} else if(DATA_MODE_WEB.equals(mode) && args.length > 0) {
doWebMode();
} else if(DATA_MODE_STDIN.equals(mode)) {
doStdinMode();
} else {
usageShort();
return;
}
if (commit) commit();
if (optimize) optimize();
final long endTime = System.currentTimeMillis();
displayTiming(endTime - startTime);
}
http://svn.apache.org/repos/asf/lucene/dev/trunk/solr/core/src/java/org/apache/solr/util/SimplePostTool.java
You could try to modify the code but I think a better bet would be to either pre-process your xml files to include the missing fields, or learn to use the API (either via Java or hitting it with Curl) to do this on your own.
Related
Hi I'm having trouble in filtering SharePoint documents through the use of Rest API since I'm using multiple lines of texts(Plain text) column to filter them out. It only returns null result after trying it out.
Single line of text column seems to work well but I need Multiple lines of text because the metadata exceeds the 255 char limit.
I'm new to SharePoint, please help. Thank you
I am using the API to pull from a multiple line column, Product.
The first part is just to keep looking if empty and I keep js old school for IE 11 users. Product is defined earlier in the js as a variable based on the page, I am using this in many ways. Basically the answer to your question is check for null and instruct to continue, then else if and use indexOf().
for (i = 0; i < data.d.results.length; i++) {
if (data.d.results[i].Product == null) {
continue;
} else if (data.d.results[i].Product.indexOf(product) !== -1) {
var xid = data.d.results[i];
insertText(xid);
}
}
The success function for the GET carries over the xid, insertText(xid). Hopefully this makes sense.
go to site settings > site permissions > anonymous access... you gotta disable the check against Client Object Model permission Requirement
I am new to jmeter & I am looking for an option to compare two files using Jmeter. Both the files are generated using Save Response to file in jmeter. Also both files contain response to a jdbc request, with 100s of values across multiple columns & rows. I am using __FileToString() function in my response assertion to compare the two files. But this fails if my file has data with some special chars. Any tips how could I handle this ? OR any other ways to compare two Jmeter created files ? I would also want to know the records that are different in both files.
I know files could be compared using a lot of other tools, but I would really want to do this using Jmeter please. Thank you!
You can do it using JSR223 Sampler and Groovy language. For example this code allows comparison of two text files:
def file1 = new File('/path/to/file1')
def file2 = new File('/path/to/file2')
def file1Lines = file1.readLines('UTF-8')
def file2Lines = file2.readLines('UTF-8')
if (file1Lines.size() != file2Lines.size()) {
SampleResult.setSuccessful(false)
SampleResult.setResponseMessage('Files size is different, omitting line-by-line compare')
} else {
def differences = new StringBuilder()
file1Lines.eachWithIndex { String file1Line, int number ->
String file2Line = file2Lines.get(number)
if (!file1Line.equals(file2Line)) {
differences.append('Difference # ').append(number).append('. Expected: ')
.append(file1Line).append('. Actual: ' + file2Line)
differences.append(System.getProperty('line.separator'))
}
}
if (differences.toString().length() > 0) {
SampleResult.setSuccessful(false)
SampleResult.setResponseMessage(differences.toString())
}
}
In case of any differences, the sampler will fail and you will see the information about deltas in the "Response Message" section:
Not that this will not necessary work for all types of files, for example, binary files.
References:
Groovy: reading files
SampleResult class JavaDoc
Apache Groovy - Why and How You Should Use It
In a Table data, I have to remove these [*,=,#,&,^,%] from table data. I should not remove these
special characters [(,),/,.,#,',",:-,_,,\] while moving data from mssql to mysql using pentaho data
integration
example:-
Gujarat is the country’s largest castor oil seed producing state. The m*ajor
month’s
Why not use regular expressions in the query you are using to retrieve the data? It will simplify your transformation!
You can use either Regex Evaluation step or String operations step in Spoon
Try them and if you still have problems comment below.
You can use a "User Defined Java Class" in Pentaho and write a regex expression with the replace function.
public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException{
Object[] r = getRow();
if (r == null) {
setOutputDone();
return false;
}
if (first)
{
first = false;
}
r = createOutputRow(r, 1);
if(get(Fields.In, "data").getString(r) != null)
{
String output= get(Fields.In, "data").getString(r);
output= output.replaceAll("[^a-zA-Z0-9\\s()/\\.#-_]+","");
get(Fields.Out, "output").setValue(r, output);
}else {
get(Fields.Out, "output").setValue(r, "");
}
putRow(data.outputRowMeta, r);
return true;
}
data is the input stream data and the cleaned output would be in "output" column. I have shared a sample code here.
Hope this will solve your issue. :)
Modifying my earlier answer to include the full code. Having said this, i have also created a plugin recently to handle special characters. You can have a look at my blog site :
Blog link: https://anotherreeshu.wordpress.com/2015/01/07/special-character-remover-clean-your-data-of-special-characters-pentaho-kettle-step-plugin/
Even though its only version 1.0.0, but you can have a try. It might also help ypu :)
I've got this code (Java, GAE):
// Much earlier:
playerKey = KeyFactory.keyToString(somePlayer.key);
// Then, later...
PersistenceManager pm = assassin.PMF.get().getPersistenceManager();
Key targetKey = KeyFactory.stringToKey(playerKey);
Query query = pm.newQuery(Player.class);
query.setFilter("__key__ == keyParam");
query.declareParameters("com.google.appengine.api.datastore.Key keyParam");
List<Player> players = (List<Player>) query.execute(targetKey); // <-- line 200
which generates this error:
javax.jdo.JDOFatalUserException: Unexpected expression type while parsing query. Are you certain that a field named __key__ exists on your object?
at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:354)
at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:252)
at myapp.Player.validPlayerWithKey(Player.java:200)
// [etc., snip]
But I'm not sure what it wants. I'm trying to search on the JDO id field, which I I thought I read had the special name __key__, in the documentation.
I've tried it with both
query.setFilter("__key__ == keyParam");
and
query.setFilter("ID == keyParam");
with the same results. So, what am I doing wrong? Or, more importantly, how do I do it correctly?
Thanks!
Edit: For completeness's sake, here is the final, working code (based on Gordon's answer, which I have accepted as correct):
Player result = null;
if (playerKey == null)
{
log.log(Level.WARNING, "Tried to find player with null key.");
}
else
{
PersistenceManager pm = assassin.PMF.get().getPersistenceManager();
try {
result = (Player) pm.getObjectById(Player.class, playerKey);
} catch (javax.jdo.JDOObjectNotFoundException notFound) {
// Player not found; we will return null.
result = null;
}
pm.close();
}
return result;
If your objective is to get an object by key, then you should use the PersistenceManager's getObjectByID() method. More details here.
As an aside, trying to construct a query to get something by it's key is something you shouldn't need to do. Although this is how you would work with an SQL database, the Google Data Store does things differently, and this is one of those cases where rather than go through the trouble of constructing a query, Google App Engine lets you get what you want directly. After all, you should only have one entity in the database with a particular key, so there's nothing in the rest of the machinery of a GQL query that you need in this case, hence it can all be skipped for efficiency.
I would recommend you to use the JPA ( http://code.google.com/appengine/docs/java/datastore/usingjpa.html ) to access your data in GAE, it has the very important advantage that you can use the widely known and documented JPA standard (and its JPAQL querying language) to do this kind of things, in portable way (if you stick to the JPA standard, your code will work for GAE, for Hibernate or with EclipseLink without modification)
I'm researching hours and hours, but I could not find any clear, efficient way to make it :/
I have a codeigniter base website in English and I have to add a Polish language now. What is the best way to make my site in 2 language depending visitor selection?
is there any way to create array files for each language and call them in view files depends on Session from lang selection? I don't wanna use database.
Appreciate helps! I'm running out of deadline :/ thanks!!
Have you seen CodeIgniter's Language library?
The Language Class provides functions
to retrieve language files and lines
of text for purposes of internationalization.
In your CodeIgniter system folder you'll
find one called language containing sets
of language files. You can create your
own language files as needed in order
to display error and other messages in
other languages.
Language files are typically stored in
your system/language directory. Alternately
you can create a folder called language
inside your application folder and store
them there. CodeIgniter will look first
in your application/language directory.
If the directory does not exist or the
specified language is not located there
CI will instead look in your global
system/language folder.
In your case...
you need to create a polish_lang.php and english_lang.php inside application/language/polish
then create your keys inside that file (e.g. $lang['hello'] = "Witaj";
then load it in your controller like $this->lang->load('polish_lang', 'polish');
then fetch the line like $this->lang->line('hello'); Just store the return value of this function in a variable so you can use it in your view.
Repeat the steps for the english language and all other languages you need.
Also to add the language to the session, I would define some constants for each language, then make sure you have the session library autoloaded in config/autoload.php, or you load it whenever you need it. Add the users desired language to the session:
$this->session->set_userdata('language', ENGLISH);
Then you can grab it anytime like this:
$language = $this->session->userdata('language');
In the controller add following lines when you make the cunstructor
i.e, after
parent::Controller();
add below lines
$this->load->helper('lang_translate');
$this->lang->load('nl_site', 'nl'); // ('filename', 'directory')
create helper file lang_translate_helper.php with following function and put it in directory system\application\helpers
function label($label, $obj)
{
$return = $obj->lang->line($label);
if($return)
echo $return;
else
echo $label;
}
for each of the language, create a directory with language abbrevation like en, nl, fr, etc., under
system\application\languages
create language file in above (respective) directory which will contain $lang array holding pairs label=>language_value as given below
nl_site_lang.php
$lang['welcome'] = 'Welkom';
$lang['hello word'] = 'worde Witaj';
en_site_lang.php
$lang['welcome'] = 'Welcome';
$lang['hello word'] = 'Hello Word';
you can store multiple files for same language with differently as per the requirement
e.g, if you want separate language file for managing backend (administrator section) you can use it in controller as $this->lang->load('nl_admin', 'nl');
nl_admin_lang.php
$lang['welcome'] = 'Welkom';
$lang['hello word'] = 'worde Witaj';
and finally
to print the label in desired language, access labels as below in view
label('welcome', $this);
OR
label('hello word', $this);
note the space in hello & word you can use it like this way as well :)
whene there is no lable defined in the language file, it will simply print it what you passed to the function label.
I second Randell's answer.
However, one could always integrate a GeoIP such as http://www.maxmind.com/app/php
or http://www.ipinfodb.com/. Then you can save the results with the codeigniter session class.
If you want to use the ipinfodb.com api You can add the ip2locationlite.class.php file to your codeigniter application library folder and then create a model function to do whatever geoip logic you need for your application, such as:
function geolocate()
{
$ipinfodb = new ipinfodb;
$ipinfodb->setKey('API KEY');
//Get errors and locations
$locations = $ipinfodb->getGeoLocation($this->input->ip_address());
$errors = $ipinfodb->getError();
//Set geolocation cookie
if(empty($errors))
{
foreach ($locations as $field => $val):
if($field === 'CountryCode')
{
$place = $val;
}
endforeach;
}
return $place;
}
For easier use CI have updated this so you can just use
$this->load->helper('language');
and to translate text
lang('language line');
and if you want to warp it inside label then use optional parameter
lang('language line', 'element id');
This will output
// becomes <label for="form_item_id">language_key</label>
For good reading
http://ellislab.com/codeigniter/user-guide/helpers/language_helper.html
I've used Wiredesignz's MY_Language class with great success.
I've just published it on github, as I can't seem to find a trace of it anywhere.
https://github.com/meigwilym/CI_Language
My only changes are to rename the class to CI_Lang, in accordance with the new v2 changes.
When managing the actual files, things can get out of sync pretty easily unless you're really vigilant. So we've launched a (beta) free service called String which allows you to keep track of your language files easily, and collaborate with translators.
You can either import existing language files (in PHP array, PHP Define, ini, po or .strings formats) or create your own sections from scratch and add content directly through the system.
String is totally free so please check it out and tell us what you think.
It's actually built on Codeigniter too! Check out the beta at http://mygengo.com/string
Follow this https://github.com/EllisLab/CodeIgniter/wiki/CodeIgniter-2.1-internationalization-i18n
its simple and clear, also check out the document # http://ellislab.com/codeigniter/user-guide/libraries/language.html
its way simpler than
I am using such code in config.php:
$lang = 'ru'; // this language will be used if there is no any lang information from useragent (for example, from command line, wget, etc...
if (!empty($_SERVER['HTTP_ACCEPT_LANGUAGE'])) $lang = substr($_SERVER['HTTP_ACCEPT_LANGUAGE'],0,2);
$tmp_value = $_COOKIE['language'];
if (!empty($tmp_value)) $lang = $tmp_value;
switch ($lang)
{
case 'ru':
$config['language'] = 'russian';
setlocale(LC_ALL,'ru_RU.UTF-8');
break;
case 'uk':
$config['language'] = 'ukrainian';
setlocale(LC_ALL,'uk_UA.UTF-8');
break;
case 'foo':
$config['language'] = 'foo';
setlocale(LC_ALL,'foo_FOO.UTF-8');
break;
default:
$config['language'] = 'english';
setlocale(LC_ALL,'en_US.UTF-8');
break;
}
.... and then i'm using usualy internal mechanizm of CI
o, almost forget! in views i using buttons, which seting cookie 'language' with language, prefered by user.
So, first this code try to detect "preffered language" setted in user`s useragent (browser). Then code try to read cookie 'language'. And finaly - switch sets language for CI-application
you can make a function like this
function translateTo($language, $word) {
define('defaultLang','english');
if (isset($lang[$language][$word]) == FALSE)
return $lang[$language][$word];
else
return $lang[defaultLang][$word];
}
Friend, don't worry, if you have any application installed built in codeigniter and you wanna add some language pack just follow these steps:
1. Add language files in folder application/language/arabic (i add arabic lang in sma2 built in ci)
2. Go to the file named setting.php in application/modules/settings/views/setting.php. Here you find the array
<?php /*
$lang = array (
'english' => 'English',
'arabic' => 'Arabic', // i add this here
'spanish' => 'Español'
Now save and run the application. It's worked fine.