I'm new to Bazel and got a question regarding Bazel macros. I'm looking for the best way to structure our build.
Is it possible to iterate over a list containing the specifics to the build rules?
For example I have a list containing srcs,deps,hdrs,name etc. This list is combined into one larger list containing all modules i want to build creating one component.
If possible can someone give a short example how this would look in code?
Thanks for your time
ok i got it:
Content of Build.bazel:
load(":macro.bzl","buildmacro")
load(":SrcList.bzl","SrcLists","CommonDependencies")
[buildmacro(
current_module_name = Module[0][0],
current_module_srcs=Module[1],
current_module_hdrs=Module[2],
current_module_deps=Module[3] + CommonDependencies,
)for Module in SrcLists]
Content of macro.bzl:
def buildmacro(current_module_name,current_module_srcs,current_module_hdrs,current_module_deps):
native.cc_library(
name = current_module_name,
deps = current_module_deps,
srcs = current_module_srcs,
hdrs = current_module_hdrs,
linkstatic = 1,
visibility = ["//visibility:public"],
)
Example of the SrcLists-file:
listofcode = [["nameofrule"]["srcfiles"]["headers"]["deps"]...]
listofcode2 = ...
SrcLists = [listofcode] + [listofcode2] ...
execute bazel build :all
Related
Newbie here ;)
I need your help ;)
I have following problem:
I want to merge two TF 2.0 - Keras Models into a new Model.
Expect for example a ResNet50V2 without the head, and retrained on new data - saved weights are provided and loaded:
resnet50v2 = ResNet50V2(weights='imagenet', include_top=False)
#model.summary()
last_layer = resnet50v2.output
x = GlobalAveragePooling2D()(last_layer)
x = Dense(768, activation='relu', name='img_dense_768')(x)
out = Dense(NUM_CLASS, activation='softmax', name='img_output_layer')(x)
resnet50v2 = Model(inputs=resnet50v2.input, outputs=out)
resnet50v2.load_weights(pathToImageModel)
Expect for example a Bert Network retrained on new data - saved weights are provided, in the same way as shown before.
Now I want to skip the last softmax layer from the two models and "merge" them into a new Model.
So I took the layer before the last layer, which names I know:
model_image_output = model_image.get_layer('img_dense_768').output
model_bert_output = model_bert.get_layer('bert_output_layer_768').output
I built two new models with these informations:
model_img = Model(model_image.inputs, model_image_output)
model_bert = Model(model_bert.inputs, model_bert_output)
So now I want to concatenate the outputs of the two models into a new one and add a new "head"
concatenate = Concatenate()([model_img.output,model_bert.output])
x = Dense(512, activation = 'relu')(concatenate)
x = Dropout(0.4)(x)
output = Dense(NUM_CLASS, activation='softmax', name='final_multimodal_output')(x)
model_concat = Model([model_image.input, model_bert.input])
So far so good, model code seemed to be valid but then my knowledge ends.
The main questions are:
Also if the last softmax layers are skipped, the loaded weights should be still available or ?
The new concatened model wants to be build, ok but this ends in this error message, which I only partially understand:
NotImplementedError: When subclassing the Model class, you should implement a call method.
Is there another way to create for example the whole ensemble network and load only the pretrained parts of it and leave the rest beeing trainable?
Am I missing something? I never did subclassing the model class? If so, it was not intended XD
May I ask kindly for some hints ?
Thanks in advance!!
Kind regards ragitagha
UPDATE:
So to find my mistake more quickly and provide the solution in a more precise way:
I had to change the lines
concatenate = Concatenate()([model_img.output,model_bert.output])
x = Dense(512, activation = 'relu')(concatenate)
x = Dropout(0.4)(x)
output = Dense(NUM_CLASS, activation='softmax', name='final_multimodal_output')(x)
model_concat = Model([model_image.input, model_bert.input])
to:
concatenate = Concatenate()([model_img.output,model_bert.output])
x = Dense(512, activation = 'relu')(concatenate)
x = Dropout(0.4)(x)
output = Dense(NUM_CLASS, activation='softmax', name='final_multimodal_output')(x)
model_concat = Model([model_image.input, model_bert.input], output)
I am working on a drupal7 site.
In Dev, I have enabled and configured simplesamlphp_auth module.
I used features and strongarm to export the configuration to code.
The downloaded feature contain:
myfeature_sso.features.defaultconfig.inc
myfeature_sso.info
myfeature_sso.module
The .inc file contains the configuration values I had put in (admin/config/people/simplesamlphp_auth) correctly
Now, in a few places, I want to replace the hard coded values with variables that change based on the environment. I set the variables at the top of .inc file using variable_get $office_ou = variable_get('office_ou', NULL); ...and . A quick example is $base_url below:
$strongarm = new stdClass();
$strongarm->disabled = FALSE;
$strongarm->api_version = 1;
$strongarm->name = 'simplesamlphp_auth_logoutgotourl';
$strongarm->value = $base_url ;
$export['simplesamlphp_auth_logoutgotourl'] = $strongarm;
When I DPM these variables, they display correct values.
But on a fresh install when I enable myfeature_sso module the value of variables are missing.
missing value from $base_url variable
Can you please point me in the right direction?
Thank you.
I found the answer:
when in features ui, do not add fields relevant to simplesamle to the defaultconfig section. if you have any there remove them.
add fields relevant to simplesaml to storongarm section
export the feature
in your_feature_name.strongarm.inc add any additional php code to function your_feature_name_strongarm()
done
My question is similar to the question of janpeter. I study the ebook by Tiller and try to simulate the example 'Architecture Driven Approach' with OpenModelica and JModelica. I tried the minimal example 'BaseSystem' in OpenModelica and it works fine. But with JModelica version 1.14 I get errors in the compiling process and my script fail. My python script is:
import matplotlib.pyplot as plt
from pymodelica import compile_fmu
from pyfmi import load_fmu
# Variables: modelName, modelFile, extraLibPath
modelName = 'BaseSystem'
modelFile = 'BaseSystem.mo'
extraLibPath = 'C:\Users\Tom\Desktop\Tiller2015a\ModelicaByExample\Architectures'
compilerOption = {'extra_lib_dirs':[extraLibPath]}
# Compile model
fmuName = compile_fmu( modelName, modelFile, compiler_options=compilerOption)
# Load model
model = load_fmu( fmuName)
# Simulate model
res = model.simulate( start_time=0.0, final_time=5.0)
# Extract interesting values
res_w = res['sensor.w']
res_y = res['setpoint.y']
tSim = res['time']
# Visualize results
fig = plt.figure(1)
ax1 = fig.add_subplot(111)
ax2 = ax1.twinx()
ax1.plot(tSim, res_w, 'g-')
ax2.plot(tSim, res_y, 'b-')
ax1.set_xlabel('t (s)')
ax1.set_ylabel('w (???)', color='g')
ax2.set_ylabel('y (???)', color='b')
plt.title('BaseSystem')
plt.legend()
plt.grid(True)
plt.show()
My problem is how to compile and simulate a model that is part of a package?
I am not a jModelica user, but I think I see some confusion in your script. You wrote:
modelName = 'BaseSystem'
modelFile = 'BaseSystem.mo'
extraLibPath = 'C:\Users\Tom\Desktop\Tiller2015a\ModelicaByExample\Architectures'
But that implies (to me) that the compiler should open the package stored at C:\Users\Tom\Desktop\Tiller2015a\ModelicaByExample\Architectures. But the top-level package is ModelicaByExample and the model you want is Architectures.BaseSystem. So I think something like this is probably more appropriate:
modelName = 'Architectures.BaseSystem'
modelFile = 'package.mo'
extraLibPath = 'C:\Users\Tom\Desktop\Tiller2015a\ModelicaByExample'
The essential point here is that you should be opening ModelicaByExample (specifically, the package.mo file in the ModelicaByExample directory). That opens the ModelicaByExample package. You need to open this package because it is the top-level package. You can't load just a sub-package (which is what it looked like you were trying to do). Then, once you've got ModelicaByExample loaded, you can ask the compiler to specifically compile Architectures.BaseSystem.
I suspect OpenModelica was "helping" you by opening the top-level package anyway, even if you were asking it to open the sub-package.
But again, I don't know jModelica very well and I have definitely not tested any of these suggestions.
Thank you Michael Tiller. With your support I found the solution.
First, the modelName has to be full qualified. Second, as you mentioned, the extraLibPath should end at the top-level directory of the library ModelicaByExample. But then I got errors, that JModelica couldn't find components or declarations which are part of the Modelica Standard Library (MSL).
So I added the modelicaLibPath to the MSL, but the error messages remained the same. After many attempts, I launched the command line with administrator privileges and any errors were gone.
Here is the executable python script: BaseSystem.py
### Attention!
# The script and/or the command line must be
# started with administrator privileges
import matplotlib.pyplot as plt
from pymodelica import compile_fmu
from pyfmi import load_fmu
# Variables: modelName, modelFile, extraLibPath
modelName = 'Architectures.SensorComparison.Examples.BaseSystem'
modelFile = ''
extraLibPath = 'C:\Users\Tom\Desktop\Tiller2015a\ModelicaByExample'
modelicaLibPath = 'C:\OpenModelica1.9.2\lib\omlibrary\Modelica 3.2.1'
compileToPath = 'C:\Users\Tom\Desktop\Tiller2015a'
# Set the compiler options
compilerOptions = {'extra_lib_dirs':[modelicaLibPath, extraLibPath]}
# Compile model
fmuName = compile_fmu( modelName, modelFile, compiler_options=compilerOptions, compile_to=compileToPath)
# Load model
model = load_fmu( fmuName)
# Simulate model
res = model.simulate( start_time=0.0, final_time=5.0)
# Extract interesting values
res_w = res['sensor.w']
res_y = res['setpoint.y']
tSim = res['time']
# Visualize results
fig = plt.figure(1)
ax1 = fig.add_subplot(111)
ax2 = ax1.twinx()
ax1.plot(tSim, res_w, 'g-')
ax2.plot(tSim, res_y, 'b-')
ax1.set_xlabel('t (s)')
ax1.set_ylabel('sensor.w (rad/s)', color='g')
ax2.set_ylabel('setpoint.y (rad/s)', color='b')
plt.title('BaseSystem')
plt.legend()
plt.grid(True)
plt.show()
Does anyone know how to use the yaml-cpp lib in NS3 module?
I have finished the source code, but I have a hard time figuring out how to link it as it got the undefined reference to error when building with waf.
To use yaml-cpp you just have to write something like :
def options(opt):
opt.load("cxx_compiler")
def configure(conf):
conf.load("cxx_compiler")
conf.check_cfg(package = "yaml-cpp", args = ["--cflags", "--libs", ], uselib_store = "YAMLCPP")
def build(bld):
bld.program(source = "main.cpp", use = "YAMLCPP")
waf use pkg-config to get all the options. Main is the source file using yaml-cpp.
I'm looking for an elegant way to remove all files in a folder that have the extension .jpg
I have the following to count the total jpg files in a folder:
Option(new File(path).list).map(_.filter(_.endsWith(".jpg")).size).getOrElse(0)
Thanks in advance, any help much appreciated :)
for {
files <- Option(new File(path).listFiles)
file <- files if file.getName.endsWith(".jpg")
} file.delete()
Some extra comment,
To extend #Debilski's answer:
Touching files obviously causes side effects. To make this functionally effectfull, please do something like:
def deleteFilesBySuffix[G[_]: Sync](suffix: String)(dirName: String): G[Unit] =
Sync[G].suspend(Sync[G].fromTry(Try(for {
files <- Option(new File(dirName).listFiles)
file <- files if file.getName.endsWith(suffix)
} file.delete())))
Then,
You'll have to run this code with an effect which can delay the
execution of this method like:
import cats.IO
import cats.syntax.foldable._
val r = deleteFilesBySuffix[IO]("jpg")("/tmp")
//Still nothing happened
//Another example with multiple dirs:
val dirNames = List("/tmp", "/tmp/myDir")
val res = dirNames.traverse_(deleteFilesBySuffix[IO]("jpg"))
//Actually run it..
r.unsafeRunSunc()
//Now files are deleted..
In my opinion this is much safer, and uses Scala's power of effects
os-lib lets you delete all files with a certain extension with a one liner:
os.list(os.pwd/"pics").filter(_.ext == "jpg").map(os.remove)
As described here, os-lib is the easiest way to perform filesystem operations with Scala. It lets you write beautiful, concise, Scala code without dealing with ugliness of the underlying Java libs.
Here's some setup code if you'd like to test this out on your machine:
os.makeDir(os.pwd/"pics")
os.write(os.pwd/"pics"/"family.jpg", "")
os.write(os.pwd/"pics"/"cousins.txt", "")
os.write(os.pwd/"pics"/"gf.jpg", "")
os.write(os.pwd/"pics"/"friend.gif", "")