I'm exploring the potential of loop though calendar in kotlin (in Android studio). In particular I would like to perform an operation for every single day between two fixed dates.
I have seen something similar with java.time.LocalDate ( while (date.isBefore(endDate)) ..{} ).
Do you think it is possible also with Calendar?
Any suggestions would be highly appreciated.
Thank you
I have found this code which works perfectly for my case; I report it below hoping it will be useful to others as well.
class DateIterator(val startDate: LocalDate,
val endDateInclusive: LocalDate,
val stepDays: Long): Iterator<LocalDate> {
private var currentDate = startDate
override fun hasNext() = currentDate <= endDateInclusive
override fun next(): LocalDate {
val next = currentDate
currentDate = currentDate.plusDays(stepDays)
return next
}
}
class DateProgression(override val start: LocalDate,
override val endInclusive: LocalDate,
val stepDays: Long = 1) :
Iterable<LocalDate>, ClosedRange<LocalDate> {
override fun iterator(): Iterator<LocalDate> =
DateIterator(start, endInclusive, stepDays)
infix fun step(days: Long) = DateProgression(start, endInclusive, days)
}
operator fun LocalDate.rangeTo(other: LocalDate) = DateProgression(this, other)
val startDate = LocalDate.of(2021, 1, 1)
val endDate = LocalDate.of(2021, 1, 31)
for (date in startDate..endDate step 1) {
// do something
}
Related
I have a class in which I get the number of ids per year with a soql query on a custom object:
public static Integer numberRecords(Integer year) {
List<AggregateResult> numbersOfRecords = [
SELECT Keeper__c
FROM Month_Expense_Application__c
WHERE calendar_year(MonthDate__c) =: year
];
Set<Id> keepers = new Set<Id>();
for (AggregateResult result : numbersOfRecords) {
keepers.add((Id)result.get('Keeper__c'));
}
Integer value = keepers.size();
return value;
}
I'm trying to write a test for this method by creating an object and filling in the fields it needs, then I try to get the id with a soql request and compare:
#isTest
public class ExpenseAdminControllerTests {
#isTest
public static void numberRecordsTests() {
Month_Expense_Application__c monthExpenseTest = new Month_Expense_Application__c(
Balance__c = 12.3,
Income__c = 11,
Keeper__c = '0034x00001K7kGCAAZ',
MonthDate__c = date.newInstance(2022, 11, 21)
);
Integer year = 2022;
List<Month_Expense_Application__c> numbersOfRecords = [
SELECT Keeper__c
FROM Month_Expense_Application__c
WHERE calendar_year(MonthDate__c) =: year AND Id =: monthExpenseTest.Id
];
Set<Id> keepers = new Set<Id>();
keepers.add(numbersOfRecords[0].Keeper__c);
Integer value = keepers.size();
System.assertEquals(1, value);
}
}
But i cant tell me what am i doing wrong
I guess you just forgot to insert "monthExpenseTest" in actual database, by means that you have just created a object by using below lines as you mentioned
Month_Expense_Application__c monthExpenseTest = new Month_Expense_Application__c(
Balance__c = 12.3,
Income__c = 11,
Keeper__c = '0034x00001K7kGCAAZ',
MonthDate__c = date.newInstance(2022, 11, 21)
);
by writing above lines you just created object which is present in memory only. it is not stored in database, so that it is not fetched when you try to fetch it via SOQL Query.
Now you may want to add one more line after above code.which is given as below
insert monthExpenseTest ;
so after inserting your object is stored in database. now you can fetch it via SOQL query.
So your test class will be looks like below
#isTest
public class ExpenseAdminControllerTests {
#isTest
public static void numberRecordsTests() {
Month_Expense_Application__c monthExpenseTest = new Month_Expense_Application__c(
Balance__c = 12.3,
Income__c = 11,
Keeper__c = '0034x00001K7kGCAAZ',
MonthDate__c = date.newInstance(2022, 11, 21)
);
insert monthExpenseTest;
Integer year = 2022;
Integer keepers = ExpenseAdminController.numberRecords(year);
System.assertEquals(1, keepers);
}
}
So when you are testing ExpenseAdminController then you just need to call that method because you want to test it. in test class which is mentioned in question it not testing numberRecords() method instead of you are rewriting logic.
I am using Flink 1.12 and I have a keyed stream, in my code it looks that both A and B share the same watermark? and therefore B is determined as late because A's coming has upgraded the watermark to be 2020-08-30 10:50:11?
The output is A(2020-08-30 10:50:08, 2020-08-30 10:50:16):2020-08-30 10:50:15,there is no output for B
I would ask whether it is possible to make different keys have independent watermark? A's watermark and B'watermark change independently
The application code is:
import java.text.SimpleDateFormat
import java.util.Date
import java.util.concurrent.TimeUnit
import org.apache.flink.streaming.api.TimeCharacteristic
import org.apache.flink.streaming.api.functions.AssignerWithPunctuatedWatermarks
import org.apache.flink.streaming.api.scala.function.WindowFunction
import org.apache.flink.streaming.api.scala.{StreamExecutionEnvironment, _}
import org.apache.flink.streaming.api.watermark.Watermark
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows
import org.apache.flink.streaming.api.windowing.time.Time
import org.apache.flink.streaming.api.windowing.windows.TimeWindow
import org.apache.flink.util.Collector
object DemoDiscardLateEvent4_KeyStream {
def to_milli(str: String) =
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(str).getTime
def to_char(milli: Long) = {
val date = if (milli <= 0) new Date(0) else new Date(milli)
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(date)
}
def main(args: Array[String]): Unit = {
val env = StreamExecutionEnvironment.getExecutionEnvironment
env.setParallelism(1)
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
val data = Seq(
("A", "2020-08-30 10:50:15"),
("B", "2020-08-30 10:50:07")
)
env.fromCollection(data).setParallelism(1).assignTimestampsAndWatermarks(new AssignerWithPunctuatedWatermarks[(String, String)]() {
var maxSeen = Long.MinValue
override def checkAndGetNextWatermark(lastElement: (String, String), extractedTimestamp: Long): Watermark = {
val eventTime = to_milli(lastElement._2)
if (eventTime > maxSeen) {
maxSeen = eventTime
}
//Allow 4 seconds late
new Watermark(maxSeen - 4000)
}
override def extractTimestamp(element: (String, String), previousElementTimestamp: Long): Long = to_milli(element._2)
}).keyBy(_._1).window(TumblingEventTimeWindows.of(Time.of(8, TimeUnit.SECONDS))).apply(new WindowFunction[(String, String), String, String, TimeWindow] {
override def apply(key: String, window: TimeWindow, input: Iterable[(String, String)], out: Collector[String]): Unit = {
val start = to_char(window.getStart)
val end = to_char(window.getEnd)
val sb = new StringBuilder
//the start and end of the window
sb.append(s"$key($start, $end):")
//The content of the window
input.foreach {
e => sb.append(e._2 + ",")
}
out.collect(sb.toString().substring(0, sb.length - 1))
}
}).print()
env.execute()
}
}
While it would sometimes be helpful if Flink offered per-key watermarking, it does not.
Each parallel instance of your WatermarkStrategy (or in this case, of your AssignerWithPunctuatedWatermarks) is generating watermarks independently, based on the timestamps of the events it observes (regardless of their keys).
One way to work around the lack of this feature is to not use watermarks at all. For example, if you would be using per-key watermarks to trigger keyed event-time windows, you can instead implement your own windows using a KeyedProcessFunction, and instead of using watermarks to trigger event time timers, keep track of the largest timestamp seen so far for each key, and whenever updating that value, determine if you now want to close one or more windows for that key.
See one of the Flink training lessons for an example of how to implement keyed tumbling windows with a KeyedProcessFunction. This example depends on watermarks but should help you get started.
We have a Flink job that does intervalJoin two streams, both streams consume events from Kafka. Here is the example code
val articleEventStream: DataStream[ArticleEvent] = env.addSource(articleEventSource)
.assignTimestampsAndWatermarks(new ArticleEventAssigner)
val feedbackEventStream: DataStream[FeedbackEvent] = env.addSource(feedbackEventSource)
.assignTimestampsAndWatermarks(new FeedbackEventAssigner)
articleEventStream
.keyBy(article => article.id)
.intervalJoin(feedbackEventStream.keyBy(feedback => feedback.article.id))
.between(Time.seconds(-5), Time.seconds(10))
.process(new ProcessJoinFunction[ArticleEvent, FeedbackEvent, String] {
override def processElement(left: ArticleEvent, right: FeedbackEvent, ctx: ProcessJoinFunction[ArticleEvent, FeedbackEvent, String]#Context, out: Collector[String]): Unit = {
out.collect(left.name + " got feedback: " + right.feedback);
}
});
});
class ArticleEventAssigner extends AssignerWithPunctuatedWatermarks[ArticleEvent] {
val bound: Long = 5 * 1000
override def checkAndGetNextWatermark(lastElement: ArticleEvent, extractedTimestamp: Long): Watermark = {
new Watermark(extractedTimestamp - bound)
}
override def extractTimestamp(element: ArticleEvent, previousElementTimestamp: Long): Long = {
element.occurredAt
}
}
class FeedbackEventAssigner extends AssignerWithPunctuatedWatermarks[FeedbackEvent] {
val bound: Long = 5 * 1000
override def checkAndGetNextWatermark(lastElement: FeedbackEvent, extractedTimestamp: Long): Watermark = {
new Watermark(extractedTimestamp - bound)
}
override def extractTimestamp(element: FeedbackEvent, previousElementTimestamp: Long): Long = {
element.occurredAt
}
}
However, we do not see any joined output. We checked that each stream does continuously emit elements with timestamp and proper watermark. Does anyone have any hint what could be possible reasons?
After checking different parts (timestamp/watermark, triggers), I just noticed that I made a mistake, i.e., the window size I used
between(Time.seconds(-5), Time.seconds(10))
is just too small, which could not find elements from both streams to join. This might sound obvious, but since I am new to Flink, I did not know where to check.
So, my lesson is that if the join does not output, it could be necessary to check the window size.
And thanks all for the comments!
I am implementing a SourceFunction, which reads Data from a Database.
The job should be able to be resumed if stopped or crushed (i.e savepoints and checkpoints) with the data being processed exactly once.
What I have so far:
#SerialVersionUID(1L)
class JDBCSource(private val waitTimeMs: Long) extends
RichParallelSourceFunction[Event] with StoppableFunction with LazyLogging{
#transient var client: PostGreClient = _
#volatile var isRunning: Boolean = true
val DEFAULT_WAIT_TIME_MS = 1000
def this(clientConfig: Serializable) =
this(clientConfig, DEFAULT_WAIT_TIME_MS)
override def stop(): Unit = {
this.isRunning = false
}
override def open(parameters: Configuration): Unit = {
super.open(parameters)
client = new JDBCClient
}
override def run(ctx: SourceFunction.SourceContext[Event]): Unit = {
while (isRunning){
val statement = client.getConnection.createStatement()
val resultSet = statement.executeQuery("SELECT name, timestamp FROM MYTABLE")
while (resultSet.next()) {
val event: String = resultSet.getString("name")
val timestamp: Long = resultSet.getLong("timestamp")
ctx.collectWithTimestamp(new Event(name, timestamp), timestamp)
}
}
}
override def cancel(): Unit = {
isRunning = false
}
}
How can I make sure to only get the rows of the database which aren't processed yet?
I assumed the ctx variable would have some information about the current watermark so that I could change my query to something like:
select name, timestamp from myTable where timestamp > ctx.getCurrentWaterMark
But it doesn't have any relevant methods for me. Any Ideas how to solve this problem would be appreciated
You have to implement CheckpointedFunction so that you can manage checkpointing by yourself. The documentation of the interface is pretty comprehensive but if you need an example I advise you to take a look at an example.
In essence, your function must implement CheckpointedFunction#snapshotState to store the state you need using Flink's managed state and then, when performing a restore, it will read that same state in CheckpointedFunction#initializeState.
I was wondering if it is possible to make particular days unavailable from the calendar (DatePicker), more specifically every Monday and Tuesday. I have found similar threads (How do create a DatePicker with only Sundays enabled? and Disable specific days of the week on jQuery UI datepicker) about blacking out dates, however, I have not been able to modify their code for my specific goal. I'm writing this application in VB.NET (WPF).
The functions I used so far, for blacking out dates are:
Datepicker1.BlackoutDates.AddDatesInPast()
Datepicker2.BlackoutDates.Add(New CalendarDateRange(DateTime.Now.AddDays(1), DateTime.MaxValue))
Where the first function will blackout the past-dates, and the second will black out all future dates. Because there is a 'DateRange' required for the second function, I'm not able to alter this function for my need.
Thanks in advance
Jerry
I modified one of the examples and came up with this.
It worked for me.
private void MyDatePicker_CalendarOpened(object sender, RoutedEventArgs e)
{
MyDatePicker.DisplayDateStart = DateTime.Now;
MyDatePicker.DisplayDateEnd = DateTime.Now + TimeSpan.FromDays(1000);
var minDate = MyDatePicker.DisplayDateStart ?? DateTime.MinValue;
var maxDate = MyDatePicker.DisplayDateEnd ?? DateTime.MaxValue;
for (var d = minDate; d <= maxDate && DateTime.MaxValue > d; d = d.AddDays(1))
{
if (d.DayOfWeek == DayOfWeek.Monday || d.DayOfWeek == DayOfWeek.Tuesday)
{
MyDatePicker.BlackoutDates.Add(new CalendarDateRange(d));
}
}
}
And here's a bonus: Prevent Certain Dates from Being Selected.
Thank you Okuma Scott, that was some helpful feedback! I rewrote your bit of code to VB language and according to my specific needs.
The included code will check all days in the next year, and will black out all the Mondays and Tuesdays.
Private Sub Datepicker_CalendarOpened(sender As Object, e As RoutedEventArgs) Handles Datepicker.CalendarOpened
Dim currDate As Date = DateTime.Now
Dim maxDate As Date = DateTime.Now.AddDays(356)
While (currDate < maxDate)
If currDate.DayOfWeek = DayOfWeek.Monday Or currDate.DayOfWeek = DayOfWeek.Tuesday Then
DatumSelectie.BlackoutDates.Add(New CalendarDateRange(currDate))
End If
currDate = currDate.AddDays(1)
End While
End Sub