Overriding Hystrix command properties specified in .properties file - hystrix

I need to override command timeout property specified in my application.properties file. Here is what I tried
#Test
public void testTokenQueryTimeout() throws Exception
{
String propertyToSet ="hystrix.command.quickbaseTokenQueryCommand.execution.isolation.thread.timeoutInMilliseconds";
String prop="";
try {
prop = ConfigurationManager.getConfigInstance().getProperty(
propertyToSet).toString();
logger.info("\n\n\noriginal quickbaseTokenQueryCommand timeout ="+prop);
System.setProperty(
propertyToSet,"10");
prop = ConfigurationManager.getConfigInstance().getProperty(
propertyToSet).toString();
logger.info("\n\n\nupdated quickbaseTokenQueryCommand timeout ="+prop);
String response = accountValidation.isValidToken(token);
logger.info(response);
Assert.assertFalse(true);
}
catch (AccountValidationServiceException e)
{
Assert.assertTrue(Constants.ERRCODE_TOKEN_QUERY_TIMED_OUT.equals(e.getErrorCode()));
}
finally {
ConfigurationManager.getConfigInstance().clearProperty(propertyToSet);
System.clearProperty(propertyToSet);
if(!GeneralUtil.isObjectEmpty(System.getProperty(
propertyToSet)))prop = System.getProperty(
propertyToSet);
logger.info("Updated testTokenQueryTimeout timeout ="+prop);
}
}
Notice, System.setProperty(propertyToSet,"10"). With this approach this test case passes i.e. the property gets changed and command times out but another test case fails due to this command timeout though I am clearing the property from System.
I also tried setting the property using ConfigurationManager.getConfigInstance().setProperty(
propertyToSet).toString(),"10"); But in that case, this change of property has no effect and command does not timeout.
Is there something I am missing here.
Please help.

Try using the ConcurrentCompositeConfiguration class
application.properties
hystrix.command.HelloWorldCommand.execution.isolation.thread.timeoutInMilliseconds=200
Command
public class HelloWorldCommand extends HystrixCommand<String> {
public HelloWorldCommand() {
super(HystrixCommandGroupKey.Factory.asKey("HelloWorldGroup"));
}
#Override
protected String run() throws Exception {
TimeUnit.MILLISECONDS.sleep(1100);
return "Hello";
}
}
Test
public class HelloWorldCommandTest {
#Test
public void commandConfigTest() {
String propertyKey = "hystrix.command.HelloWorldCommand.execution.isolation.thread.timeoutInMilliseconds";
ConcurrentCompositeConfiguration config = (ConcurrentCompositeConfiguration) ConfigurationManager.getConfigInstance();
Integer originalTimeout = (Integer) config.getProperty(propertyKey);
config.setOverrideProperty(propertyKey, 1200);
String result = new HelloWorldCommand().execute();
assertThat(result, is("Hello"));
config.setOverrideProperty(propertyKey, originalTimeout);
Integer timeoutValue = (Integer) config.getProperty(propertyKey);
assertThat(timeoutValue, is(originalTimeout));
}
}

Related

how to remove spring aop proxy dynamically?

Given the following Aspect defination,
#Component
#Aspect
public class DefAspect {
Logger log = Logger.getLogger("DefAspect");
#Around("execution(* com.jd.brick.service.OrderService.saveOrder(..))")
public Object around(ProceedingJoinPoint proceedingJoinPoint) throws Throwable {
long start = System.currentTimeMillis();
Object value;
try {
value = proceedingJoinPoint.proceed();
} catch (Throwable throwable) {
throw throwable;
} finally {
long duration = System.currentTimeMillis() - start;
String msg = String.format("%s.%s took %s ms",
proceedingJoinPoint.getSignature().getDeclaringType().getSimpleName(),
proceedingJoinPoint.getSignature().getName(),
duration);
log.info(msg);
}
return value;
}
}
Spring will automatically make a proxy object, which will log the duration of method invocation.
But in some situation, the generated proxy object need to be removed.
In general, spring bean can be removed from BeanDefinitionRegistry.
ConfigurableApplicationContext ctx = SpringApplication.run(ProxyApplication.class, args);
BeanDefinitionRegistry registry = (BeanDefinitionRegistry) ctx.getAutowireCapableBeanFactory();
for(String beanName : ctx.getBeanDefinitionNames()){
System.out.println(beanName);
registry.removeBeanDefinition(beanName);
}
As for AOP situation, the bean name seems not easy determined.
Any help would be appreciated, thanks in advance.

BroadcastProcessFunction Processing Delay

I'm fairly new to Flink and would be grateful for any advice with this issue.
I wrote a job that receives some input events and compares them with some rules before forwarding them on to kafka topics based on whatever rules match. I implemented this using a flatMap and found it worked well, with one downside: I was loading the rules just once, during application startup, by calling an API from my main() method, and passing the result of this API call into the flatMap function. This worked, but it means that if there are any changes to the rules I have to restart the application, so I wanted to improve it.
I found this page in the documentation which seems to be an appropriate solution to the problem. I wrote a custom source to poll my Rules API every few minutes, and then used a BroadcastProcessFunction, with the Rules added to to the broadcast state using processBroadcastElement and the events processed by processElement.
The solution is working, but with one problem. My first approach using a FlatMap would process the events almost instantly. Now that I changed to a BroadcastProcessFunction each event takes 60 seconds to process, and it seems to be more or less exactly 60 seconds every time with almost no variation. I made no changes to the rule matching logic itself.
I've had a look through the documentation and I can't seem to find a reason for this, so I'd appreciate if anyone more experienced in flink could offer a suggestion as to what might cause this delay.
The job:
public static void main(String[] args) throws Exception {
// set up the streaming execution environment
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.IngestionTime);
// read the input from Kafka
DataStream<KafkaEvent> documentStream = env.addSource(
createKafkaSource(getSourceTopic(), getSourceProperties())).name("Kafka[" + getSourceTopic() + "]");
// Configure the Rules data stream
DataStream<RulesEvent> ruleStream = env.addSource(
new RulesApiHttpSource(
getApiRulesSubdomain(),
getApiBearerToken(),
DataType.DataTypeName.LOGS,
getRulesApiCacheDuration()) // Currently set to 120000
);
MapStateDescriptor<String, RulesEvent> ruleStateDescriptor = new MapStateDescriptor<>(
"RulesBroadcastState",
BasicTypeInfo.STRING_TYPE_INFO,
TypeInformation.of(new TypeHint<RulesEvent>() {
}));
// broadcast the rules and create the broadcast state
BroadcastStream<RulesEvent> ruleBroadcastStream = ruleStream
.broadcast(ruleStateDescriptor);
// extract the resources and attributes
documentStream
.connect(ruleBroadcastStream)
.process(new FanOutLogsRuleMapper()).name("FanOut Stream")
.addSink(createKafkaSink(getDestinationProperties()))
.name("FanOut Sink");
// run the job
env.execute(FanOutJob.class.getName());
}
The custom HTTP source which gets the rules
public class RulesApiHttpSource extends RichSourceFunction<RulesEvent> {
private static final Logger LOGGER = LoggerFactory.getLogger(RulesApiHttpSource.class);
private final long pollIntervalMillis;
private final String endpoint;
private final String bearerToken;
private final DataType.DataTypeName dataType;
private final RulesApiCaller caller;
private volatile boolean running = true;
public RulesApiHttpSource(String endpoint, String bearerToken, DataType.DataTypeName dataType, long pollIntervalMillis) {
this.pollIntervalMillis = pollIntervalMillis;
this.endpoint = endpoint;
this.bearerToken = bearerToken;
this.dataType = dataType;
this.caller = new RulesApiCaller(this.endpoint, this.bearerToken);
}
#Override
public void open(Configuration configuration) throws Exception {
// do nothing
}
#Override
public void close() throws IOException {
// do nothing
}
#Override
public void run(SourceContext<RulesEvent> ctx) throws IOException {
while (running) {
if (pollIntervalMillis > 0) {
try {
RulesEvent event = new RulesEvent();
event.setRules(getCurrentRulesList());
event.setDataType(this.dataType);
event.setRetrievedAt(Instant.now());
ctx.collect(event);
Thread.sleep(pollIntervalMillis);
} catch (InterruptedException e) {
running = false;
}
} else if (pollIntervalMillis <= 0) {
cancel();
}
}
}
public List<Rule> getCurrentRulesList() throws IOException {
// call API and get rulles
}
#Override
public void cancel() {
running = false;
}
}
The BroadcastProcessFunction
public abstract class FanOutRuleMapper extends BroadcastProcessFunction<KafkaEvent, RulesEvent, KafkaEvent> {
protected final String RULES_EVENT_NAME = "rulesEvent";
protected final MapStateDescriptor<String, RulesEvent> ruleStateDescriptor = new MapStateDescriptor<>(
"RulesBroadcastState",
BasicTypeInfo.STRING_TYPE_INFO,
TypeInformation.of(new TypeHint<RulesEvent>() {
}));
#Override
public void processBroadcastElement(RulesEvent rulesEvent, BroadcastProcessFunction<KafkaEvent, RulesEvent, KafkaEvent>.Context ctx, Collector<KafkaEvent> out) throws Exception {
ctx.getBroadcastState(ruleStateDescriptor).put(RULES_EVENT_NAME, rulesEvent);
LOGGER.debug("Added to broadcast state {}", rulesEvent.toString());
}
// omitted rules matching logic
}
public class FanOutLogsRuleMapper extends FanOutRuleMapper {
public FanOutLogsJobRuleMapper() {
super();
}
#Override
public void processElement(KafkaEvent in, BroadcastProcessFunction<KafkaEvent, RulesEvent, KafkaEvent>.ReadOnlyContext ctx, Collector<KafkaEvent> out) throws Exception {
RulesEvent rulesEvent = ctx.getBroadcastState(ruleStateDescriptor).get(RULES_EVENT_NAME);
ExportLogsServiceRequest otlpLog = extractOtlpMessageFromJsonPayload(in);
for (Rule rule : rulesEvent.getRules()) {
boolean match = false;
// omitted rules matching logic
if (match) {
for (RuleDestination ruleDestination : rule.getRulesDestinations()) {
out.collect(fillInTheEvent(in, rule, ruleDestination, otlpLog));
}
}
}
}
}
Maybe you can give the complete code of the FanOutLogsRuleMapper class, currently the match variable is always false

Flink Streaming RichSource early stops

It runs with processing time and using a broadcast state.
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.ProcessingTime);
BroadcastStream<List<TableOperations>> broadcastOperationsState = env
.addSource(new LoadCassandraOperations(10000L, cassandraHost, cassandraPort)).broadcast(descriptor);
SingleOutputStreamOperator<InternalVariableValue> stream =
env.addSource(new SourceMillisInternalVariableValue(5000L));
SingleOutputStreamOperator<InternalVariableOperation> streamProcessed =
stream
.keyBy(InternalVariableValue::getUuid)
.connect(broadcastOperationsState)
.process(new AddOperationInfo())
;
streamProcessed.print();
SourceMillisIntervalVariableValues create a event every 5s . The events are stored in a static collection. The run method looks like :
public class SourceMillisInternalVariableValue extends RichSourceFunction<InternalVariableValue>{
private boolean running;
long millis;
public SourceMillisInternalVariableValue(long millis) {
super();
this.millis = millis;
}
#Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
running = true;
}
#Override
public void cancel() {
running = false;
}
#Override
public void run(SourceContext<InternalVariableValue> ctx) throws Exception {
//Espera inicial
Thread.sleep(1500);
PojoVariableValues[] pojoData =
new PojoVariableValues[]{
new PojoVariableValues("id1", "1"),
new PojoVariableValues("id2", "2"),
....
....
new PojoVariableValues("id21", "21")
};
int cont = 0;
while (cont<pojoData.length) {
System.out.println("Iteration "+cont+" "+pojoData.length);
ctx.collect(generateVar(pojoData[0+cont].getUUID(), pojoData[0+cont].getValue()));
ctx.collect(generateVar(pojoData[1+cont].getUUID(), pojoData[1+cont].getValue()));
ctx.collect(generateVar(pojoData[2+cont].getUUID(), pojoData[2+cont].getValue()));
cont = cont +3;
Thread.sleep(millis);
}
}
private InternalVariableValue generateVar(String uuid, String value)
{
return InternalVariableValueMessage.InternalVariableValue.newBuilder()
.setUuid(uuid)
.setTimestamp(new Date().getTime()).setValue(value).setKeyspace("nest").build();
}
class PojoVariableValues {
private String UUID;
private String Value;
public PojoVariableValues(String uUID, String value) {
super();
UUID = uUID;
Value = value;
}
public String getUUID() {
return UUID;
}
public void setUUID(String uUID) {
UUID = uUID;
}
public String getValue() {
return Value;
}
public void setValue(String value) {
Value = value;
}
}
}
LoadCassandraOperations emits events every 10 seconds. It works fine.
When I run this code, SourceMillisIntervalVariableValues stops in the first iteration, emiting only three events. If I comment the process function, both sources works properly, but if I run the process , the source is cancel...
I spect than the source emits all events ( 21 exactly ) , and all of them are processing in the aggregate function. If I run this code, the while loop in the sources only complete one iteration.
Any idea ?
Thank youuu . cheers
EDIT:
Important. This code is for explore the processint time and broadcast feature. I know that I'm not using the best practices in the sources. Thanks
EDIT 2:
The problem starts when I try to run the process function.
Solved !!
The problem was that I try to run it using a TestContainer and I can't watch any logs.
I ran it with a simple main method and I can see some code errors ( like the commented in the comments. Tnks !!! ).

Hystrix fallback handling 2 dbcall

Hystrix fallback issue.
If dbcall1 goes in fallback due to some failure and short-circuit happens, then dbcall2 also goes in fallback evrytime untill the circuit opens.
public class CommandHelloFailure extends HystrixCommand<String> {
private final String name;
public CommandHelloFailure(String name) {
super(HystrixCommandGroupKey.Factory.asKey("ExampleGroup"));
this.name = name;
}`enter code here`
String dbcall1 ()
{
new CommandHelloFailure().execute();
}
String dbcall2()
{
new CommandHelloFailure().execute();
}
#Override
protected String run() {
throw new RuntimeException("this command always fails");
}
#Override
protected String getFallback() {
return "Hello Failure " + name + "!";
}
}
However i want independent fallback for both the method(DB call).
That's because you didn't specify HystrixCommandKey. If you doesn't specify HystrixCommandKey, HystrixCommandKey is derived from the class name. Namely CommandHelloFailure will be used as HystrixCommandKey.
It means that dbcall1 and dbcall2 will share the same circuit breaker.
Because Circuit Break is created for each HystrixCommandKey, not HystrixCommandGroupKey.

CXF interceptor to preprocess incoming message

I have an operation that has an input message like this:
InputMessageType
messageType: String
other properties ...
I'd like to modify this messageType before it hits the target WS method and I wrote an interceptor for the Phase.USER_LOGICAL. However, in the handleMessage if I try to do:
message.getContent(InputMessageType.class) it returns null.
How could I get the reference to the InputMessageType, change it's messageType property and then let CXF call the WS with the modified input parameter?
I had the same problem as yours : message.getContent(xxx.class) returns null. I do not know why and I will check later this behaviour.
So instead I use the interceptor like this (I retrieve the MessageContentsList) :
public class ApiSoapActionInInterceptorService extends AbstractPhaseInterceptor<Message> {
public ApiSoapActionInInterceptorService(){
super(Phase.PRE_INVOKE);
}
#Override
public void handleMessage(Message message) throws Fault {
MessageContentsList inObjects = MessageContentsList.getContentsList(message);
if (inObjects != null && !inObjects.isEmpty()){
for (Iterator<Object> it = inObjects.iterator(); it.hasNext() ;){
Object ob = it.next();
if (ob instanceof InputMessageType){
//TODO
}
}
} else {
//TODO
}
}
}

Resources