I cannot get the value of an array in an ini file.
Here's the ini file:
module.name = Core
module.version = 1
module.package = 'Core Modules'
module.dependency[] = Dep1
module.dependency[] = Dep2
module.dependency[] = Dep3
Here's the code I use to parse it:
$ini = new Zend_Config_Ini('/path/to/module.ini');
The following works fine:
echo $ini->module->name;
This, however, causes an error ('Call to a member function toArray() on a non-object'):
$ini->module->dependency->toArray();
Also, this returns null:
var_dump($ini->module->dependency);
If I change the ini file to:
module.name = Core
module.version = 1
module.package = 'Core Modules'
dependency[] = Dep1
dependency[] = Dep2
dependency[] = Dep3
I can access the array by using:
$ini->dependency->toArray();
I want the 'module.' prefix, however, because other config data will be in the file.
Any help is greatly appreciated!
You should specify a section in the top of the config. Something like this:
[production]
module.dependency[] = Dep1
module.dependency[] = Dep2
module.dependency[] = Dep3
Now this will do ok:
$ini = new Zend_Config_Ini('/path/to/module.ini', 'production');
$ini->module->dependency->toArray();
Related
I'm really new to Flume. I prefer Flume than Sqoop because data is continued to be imported to MS SQL Server in my case, therefore I think Flume is a better choice which is able to transfer data in real time.
I just followed some online example and then editing my own flume config file which tells something about the source, channel, and sink. However, it seemed that Flume didn't work successfully. There was no data being transferred to HBase.
mssql-hbase.conf
# source, channel, sink
agent1.sources = src1
agent1.channels = ch1
agent1.sinks = sk1
# declare source type
agent1.sources.src1.type = org.keedio.flume.source.SQLSource
agent1.sources.src1.hibernate.connection.url = jdbc:sqlserver://xx.xx.xx.xx:1433;DatabaseName=xxxx
agent1.sources.src1.hibernate.connection.user = xxxx
agent1.sources.src1.hibernate.connection.password = xxxx
agent1.sources.src1.table = xxxx
agent1.sources.src1.hibernate.connection.autocommit = true
# declare mysql hibernate dialect
agent1.sources.src1.hibernate.dialect = org.hibernate.dialect.SQLServerDialect
agent1.sources.src1.hibernate.connection.driver_class = com.microsoft.sqlserver.jdbc.SQLServerDriver
#agent1.sources.src1.hibernate.provider_class=org.hibernate.connection.C3P0ConnectionProvider
#agent1.sources.src1.columns.to.select = *
#agent1.sources.src1.incremental.column.name = PK, name, machine, time
#agent1.sources.src1.start.from=0
#agent1.sources.src1.incremental.value = 0
# query time interval
agent1.sources.src1.run.query.delay = 5000
# declare the folder loaction where flume state is saved
agent1.sources.src1.status.file.path = /home/user/flume-source-state
agent1.sources.src1.status.file.name = src1.status
agent1.sources.src1.batch.size = 1000
agent1.sources.src1.max.rows = 1000
agent1.sources.src1.delimiter.entry = |
# set the channel to memory mode
agent1.channels.ch1.type = memory
agent1.channels.ch1.capacity = 10000
agent1.channels.ch1.transactionCapacity = 10000
agent1.channels.ch1.byteCapacityBufferPercentage = 20
agent1.channels.ch1.byteCapacity = 800000
# declare sink type
agent1.sinks.sk1.type = org.apache.flume.sink.hbase.HBaseSink
agent1.sinks.sk1.table = yyyy
agent1.sinks.sk1.columnFamily = yyyy
agent1.sinks.sk1.hdfs.batchSize = 100
agent1.sinks.sk1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
agent1.sinks.sk1.serializer.regex = ^\"(.*?)\",\"(.*?)\",\"(.*?)\"$
agent1.sinks.sk1.serializer.colNames = PK, name, machine, time
# bind source, channel, sink
agent1.sources.src1.channels = ch1
agent1.sinks.sk1.channel = ch1
But, I use a similar config file to transfer data from MySql to HBase. Luckily, it worked.
mysql-hbase.conf
# source, channel, sink
agent1.sources = src1
agent1.channels = ch1
agent1.sinks = sk1
# declare source type
agent1.sources.src1.type = org.keedio.flume.source.SQLSource
agent1.sources.src1.hibernate.connection.url = jdbc:mysql://xxxx:3306/userdb
agent1.sources.src1.hibernate.connection.user = xxxx
agent1.sources.src1.hibernate.connection.password = xxxx
agent1.sources.src1.table = xxxx
agent1.sources.src1.hibernate.connection.autocommit = true
# declare mysql hibernate dialect
agent1.sources.src1.hibernate.dialect = org.hibernate.dialect.MySQL5Dialect
agent1.sources.src1.hibernate.connection.driver_class = com.mysql.jdbc.Driver
#agent1.sources.src1.hibernate.provider_class=org.hibernate.connection.C3P0ConnectionProvider
#agent1.sources.src1.columns.to.select = *
#agent1.sources.src1.incremental.column.name = id
#agent1.sources.src1.incremental.value = 0
# query time interval
agent1.sources.src1.run.query.delay = 5000
# declare the folder loaction where flume state is saved
agent1.sources.src1.status.file.path = /home/user/flume-source-state
agent1.sources.src1.status.file.name = src1.status
#agent1.sources.src1.interceptors=i1
#agent1.sources.src1.interceptors.i1.type=search_replace
#agent1.sources.src1.interceptors.i1.searchPattern="
#agent1.sources.src1.interceptors.i1.replaceString=,
# Set the channel to memory mode
agent1.channels.ch1.type = memory
agent1.channels.ch1.capacity = 10000
agent1.channels.ch1.transactionCapacity = 10000
agent1.channels.ch1.byteCapacityBufferPercentage = 20
agent1.channels.ch1.byteCapacity = 800000
# declare sink type
agent1.sinks.sk1.type = org.apache.flume.sink.hbase.HBaseSink
agent1.sinks.sk1.table = user_test_2
agent1.sinks.sk1.columnFamily = user_hobby
agent1.sinks.sk1.hdfs.batchSize = 100
agent1.sinks.sk1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
agent1.sinks.sk1.serializer.regex = ^\"(.*?)\",\"(.*?)\",\"(.*?)\",\"(.*?)\"$
agent1.sinks.sk1.serializer.colNames = id,name,age,hobby
# bind source, channel, sink
agent1.sources.src1.channels = ch1
agent1.sinks.sk1.channel = ch1
Does anyone know is there something wrong in the config file? Thanks.
I want to add some extra information into the exists abc file or if its possible while creating alembic cache with some extra information in maya or any cg application using pyhon.
I am appreciate any one can help me to edit the alembic file.
input example
meta_data = {'name': 'Hero', 'tag': 'test_show'}
abc_file = '/show/test_show/scene/hero.abc'
set meta data ?
from alembic import Abc
get meta data
from alembic import Abc
archive = Abc.IArchive(abc_file)
top = archive.getTop()
meta_data = top.getMetaData()
print meta_data__str()
Here's a complete script that does the job of copying the source alembic file and inserting some metadata:
import os
import alembic
def copy_props(i_props, o_props):
'''
Copy properties
'''
for index in range(i_props.getNumProperties()):
header = i_props.getPropertyHeader(index)
if header.isArray():
i_prop = alembic.Abc.IArrayProperty(
i_props,
header.getName())
prop_name = i_prop.getName()
prop_meta = i_prop.getMetaData()
o_prop = alembic.Abc.OArrayProperty(
o_props,
prop_name,
i_prop.getDataType(),
prop_meta,
0)
o_prop.setTimeSampling(i_prop.getTimeSampling())
for i in range(i_prop.getNumSamples()):
o_prop.setValue(i_prop.getValue(i))
elif header.isScalar():
i_prop = alembic.Abc.IScalarProperty(
i_props,
header.getName())
prop_name = i_prop.getName()
prop_meta = i_prop.getMetaData()
o_prop = alembic.Abc.OScalarProperty(
o_props,
prop_name,
i_prop.getDataType(),
prop_meta,
0)
o_prop.setTimeSampling(i_prop.getTimeSampling())
for i in range(i_prop.getNumSamples()):
o_prop.setValue(i_prop.getValue(i))
elif header.isCompound():
i_prop = alembic.Abc.ICompoundProperty(
i_props,
header.getName())
prop_name = i_prop.getName()
prop_meta = i_prop.getMetaData()
o_prop = alembic.Abc.OCompoundProperty(
o_props,
prop_name,
prop_meta)
copy_props(i_prop, o_prop)
def copy_object(i_obj, o_obj):
'''
Recursively copy object data
'''
if o_obj is None:
return
i_props = i_obj.getProperties()
o_props = o_obj.getProperties()
copy_props(i_props, o_props)
for index in range(i_obj.getNumChildren()):
i_child = i_obj.getChild(index)
i_child_name = i_child.getName()
i_child_meta = i_child.getMetaData()
o_child = alembic.Abc.OObject(o_obj, i_child_name, i_child_meta)
copy_object(i_child, o_child)
def copy_abc(i_path, o_path, app, description):
'''
Copy alembic file from i_path to o_path
'''
arc_in = alembic.Abc.IArchive(i_path)
arc_out = alembic.Abc.OArchive(o_path, asOgawa=True)
arc_out = alembic.Abc.CreateArchiveWithInfo(o_path, app, description)
top_in = arc_in.getTop()
top_out = arc_out.getTop()
copy_object(top_in, top_out)
def read(abc_file):
archive = alembic.Abc.IArchive(abc_file)
return alembic.Abc.GetArchiveInfo(archive)
if __name__ == '__main__':
i_path = os.path.join(
os.path.dirname(__file__),
'itest.abc'
)
o_path = os.path.join(
os.path.dirname(__file__),
'otest.abc'
)
copy_abc(i_path, o_path, 'Cool app', 'Cool description')
print('Created archive: ' + o_path)
archive_info = read(o_path)
print('App name: ' + archive_info.get('appName'))
print('Description: ' + archive_info.get('userDescription'))
print('Written: ' + archive_info.get('whenWritten'))
You can't write just arbitrary data but you can set description and application strings:
from alembic import Abc
MY_APP = 'My cool application'
def write(abc_file, description):
archive = Abc.CreateArchiveWithInfo(abc_file, MY_APP, description)
def read(abc_file):
archive = Abc.IArchive(abc_file)
top = archive.getTop()
return Abc.GetArchiveInfo(archive)
abc_file = 'alembic.abc'
write(abc_file, 'An abc file cool description')
archive_info = read(abc_file)
print(archive_info.get('appName'))
print(archive_info.get('userDescription'))
print(archive_info.get('whenWritten'))
In Waf how can I create multiple custom tasks, that can run parallel (with --jobs=JOBS)?
Sources = ["C:\\src1.c", "C:\\Mod1\src2.c", ... 30pcs] # one per call
Incl_Paths = ["Mod1". "Mod1"] # list all of them in all call
INCL_ST = "-I%s" # how to format an include path in an argument
Ext_out = "_loc" # output file extension
The goal:
C:\\LOC.exe -IMod1 -IMod2 C:\\src1.c > build\\src1.c_loc //or better src1_loc
C:\\LOC.exe -IMod1 -IMod2 C:\\Mod1\src2.c > build\\src2.c_loc //or better src2_loc
...
I couldn't get it work
def build(bld):
for i in Sources:
bld.new_task_gen(
source = i,
rule='C:\\LOC.exe ${INCL_ST:Incl_Paths} ${SRC} > ' + i + Ext_out,
)
Also I couldn't extract the exe
# find_program(self, filename, path_list=[], var=None, environ=None, exts=''):
cfg.find_program("C:\\LOC.exe", var='LOC')
To change from:
rule='C:\\LOC.exe ...'
To:
rule='${LOC} ...'
Something like this should work with waf 1.7:
from waflib.Task import Task
from waflib.TaskGen import extension
Ext_out = "_loc" # output file extension
def configure(conf):
# loc.exe must be in the system path for this to work
conf.find_program(
'loc',
var = "LOC",
)
conf.env.Incl_Paths = ["Mod1", "Mod1"]
conf.env.INCL_ST = "-I%s"
#extension('.c')
def process_loc(self, node):
out_node = node.change_ext(Ext_out)
tsk = self.create_task('loc')
tsk.set_inputs(node)
tsk.set_outputs(out_node)
class loc_task(Task):
ext_in = ['.c']
ext_out = ['_loc']
run_str = "${LOC} ${INCL_ST:Incl_Paths} ${SRC} > ${TGT}"
def build(bld):
bld(source = ["src1.c", "src2.c"])
Well it works for me on linux faking loc ...
I have the structure that is demonstrated below:
I have a database folder which contains brands. Each brand consists of logo and query.
I want to traverse on all the the files (file_1 to file_n) in all the database and perform some operations on them.
I wrote this code:
d = dir(database);
isub = [d(:).isdir];
brandsFolders = {d(isub).name}';
brandsFolders(ismember(brandsFolders,{'.','..'})) = [];
[numberOfBrands not_used]=size(brandsFolders); %holds the number of the brands
for i=1:numberOfBrands
temp=strcat(database, '\');
currentBrand=strcat(temp, brandsFolders(i));
d = dir(currentBrand{1,1});
isub = [d(:).isdir];
logoAndQuery = {d(isub).name}';
logoAndQuery(ismember(logoAndQuery,{'.','..'})) = [];
logo=strcat(currentBrand, '\', logoAndQuery(1));
files=dir(logo{1,1});
[numberOfFiles not_used]=size(files);
for j=1:numberOfFiles
if strcmp(files(j).name, '..')~=1 && strcmp(files(j).name, '.')~=1
%operations on each files(j).name
end
end
end
The code works fine, it traverse on the desired files.
However, the code is little bit ugly and confusing.
I was wondering if I can do it in another better way?
Traversing through a set of directories goes pretty much as you are doing. However imo, there are some things you can do easier / I would do differently:
brandsFolders = dir(database);
brandsFolders( ~[brandsFolders.isdir] | strcmp({brandsFolders.name},'.') | strcmp({brandsFolders.name},'..')) = [];
for ii=1:numel(brandsFolders)
logoAndQuery = dir(fullfile(database,brandsFolders(ii).name));
logoAndQuery( ~[logoAndQuery.isdir] | strcmp({logoAndQuery.name},'.') | strcmp({logoAndQuery.name},'..')) = [];
logo = fullfile(databasecurrentBrand,brandsFolders(ii).name), logoAndQuery(1).name);
files = dir(logo);
files(strcmp({files.name},'.') | strcmp({files.name},'..'))=[];
for jj=1:numel(files)
%operations on each files(j).name
end
end
(This of course only works if you're sure that logoAndQuery(1) will always be the 'logo' directory.)
or alternatively use a subfunction for the dir-querying:
function dirs = getDirs(strPath)
dirs = dir(strPath);
dirnames = {dirs.name};
dirs ( ~[dirs.isdir] | strcmp(dirnames ,'.') | strcmp(dirnames ,'..')) = [];
end
which gives you already some shorter code and gives the following, in which I also assume there are no directories in the 'logo' directories:
brandsFolders = getDirs(database);
for ii=1:numel(brandsFolders)
logoAndQuery = getDirs(fullfile(database,brandsFolders(ii).name));
logo = fullfile(databasecurrentBrand,brandsFolders(ii).name), logoAndQuery(1).name);
files = dir(logo);
files([files.isdir])=[];
for jj=1:numel(files)
%operations on each files(j).name
end
end
I'm trying to synchronize OpenLDAP and Active directory together. To do so I'm using a program called LSC-Project which is specified to do this sort of thing.
I have configured the program the best I can however I can't find a way to shake off the following error:
javax.naming.NameNotFoundException: [LDAP: error code 32 - 0000208D: NameErr: DSID-
031001CD,
problem 2001 (NO_OBJECT), data 0, best match of:
'DC=domname,DC=com'
]; remaining name
'uid=user1,ou=Users'
May 09 15:19:25 - ERROR - Error while synchronizing ID uid=user1,ou=Users:
java.lang.Exception:
Technical problem while applying modifications to directory
dn: uid=user1,ou=Users,dc=domname,dc=com
changetype: add
userPassword: 3+kU2th/WMo/v553A24a3SBw2kU=
objectClass: uid
This is the configuration file that the program runs on:
###############################
Destination LDAP directory #
##############################
dst.java.naming.provider.url = ldap://192.168.1.3:389/dc=Windows,dc=com
dst.java.naming.security.authentication = simple
dst.java.naming.security.principal = cn=Administrator,cn=Users,dc=Windows,dc=com
dst.java.naming.security.credentials = 11111
dst.java.naming.referral = ignore
dst.java.naming.ldap.derefAliases = never
dst.java.naming.factory.initial = com.sun.jndi.ldap.LdapCtxFactory
dst.java.naming.ldap.version = 3
dst.java.naming.ldap.pageSize = 1000
#########################
Source LDAP directory
#########################
src.java.naming.provider.url = ldap://192.168.1.2:389/dc=Linux,dc=com
src.java.naming.security.authentication = simple
src.java.naming.security.principal = uid=root,ou=users,dc=Linux,dc=com
src.java.naming.security.credentials = 11111
src.java.naming.referral = ignore
src.java.naming.ldap.derefAliases = never
src.java.naming.factory.initial = com.sun.jndi.ldap.LdapCtxFactory
src.java.naming.ldap.version = 3
#######################
Tasks configuration
#######################
lsc.tasks = Administrator
lsc.tasks.Administrator.srcService = org.lsc.jndi.SimpleJndiSrcService
lsc.tasks.Administrator.srcService.baseDn = ou=users
lsc.tasks.Administrator.srcService.filterAll = (&(objectClass=person))
lsc.tasks.Administrator.srcService.pivotAttrs = uid
lsc.tasks.Administrator.srcService.filterId = (&(objectClass=person)(uid={uid}))
lsc.tasks.Administrator.srcService.attrs = description uid userPassword
lsc.tasks.Administrator.dstService = org.lsc.jndi.SimpleJndiDstService
lsc.tasks.Administrator.dstService.baseDn = cn=Users
lsc.tasks.Administrator.dstService.filterAll = (&(cn=*)(objectClass=organizationalPerson))
lsc.tasks.Administrator.dstService.pivotAttrs = cn, top, person, user, organizationalPerson
lsc.tasks.Administrator.dstService.filterId = (&(objectClass=user) (sAMAccountName={cn}))
lsc.tasks.Administrator.dstService.attrs = description cn userPassword objectClass
lsc.tasks.Administrator.bean = org.lsc.beans.SimpleBean
lsc.tasks.Administrator.dn = "uid=" + srcBean.getAttributeValueById("uid") + ",ou=Users"
dn.real_root = dc=Domname,dc=com
#############################
Syncoptions configuration
#############################
lsc.syncoptions.Administrator = org.lsc.beans.syncoptions.PropertiesBasedSyncOptions
lsc.syncoptions.Administrator.default.action = M
lsc.syncoptions.Administrator.objectClass.action = M
lsc.syncoptions.Administrator.objectClass.force_value = srcBean.getAttributeValueById("cn").toUpperCase()
lsc.syncoptions.Administrator.userPassword.default_value = SecurityUtils.hash(SecurityUtils.HASH_SHA1, "defaultPassword")
lsc.syncoptions.Administrator.default.delimiter=;
lsc.syncoptions.Administrator.objectClass.force_value = "top";"user";"person";"organizationalPerson"
lsc.syncoptions.Administrator.userPrincipalName.force_value = srcBean.getAttributeValueById("uid") + "#Domname.com"
lsc.syncoptions.Administrator.userAccountControl.create_value = AD.userAccountControlSet ( "0", [AD.UAC_SET_NORMAL_ACCOUNT])
I'm suspecting that it has something to do with the baseDn of the Task configuration in the part of the source configuration.
The OSs is ubuntu 10.04 and Windows2K3
Someone suggested to me to make a manual sync between them but I have not found any guides to do so. And this program is pretty much the only thing that says that is does this kind of job without costs.
The baseDn should be the distinguished name of the base object of the search, for example, ou=users,dc=domname,dc=com.
see also
LDAP: Mastering Search Filters
LDAP: Search best practices
LDAP: Programming practices
The main reason for NameNotFoundException is that the object which you're searching doesn't exist or the container in which you are searching is not correct.
In case of Spring-ldap, we used to get this error when we specify the baseDn in the context file(LdapContextSource bean) and also in createUser code to build userDn.we need not specify the dc again in the buildUserDn()
protected Name buildUserDn(String userName) {
DistinguishedName dn = new DistinguishedName();
//only cn is required as the base dn is already specified in context file
dn.add("cn", userName);
return dn;
}
In Active Directory: Users catalog is container class, not OrganizationalUnit, so you should use: cn=users,dc=domname,dc=com