Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
I'm using ZenHub for task management. In ZenHub, I see 5 columns as displayed below:
So, I decided to move all the issues to Backlog. After that, I have a question: what are differences between To Do and In Progress ? Because all tasks in Backlog must be done. And when I want to do a task, why shouldn't I move directly through In Progress ?
You can see here a simple definition of backlog in Agile Project management:
A backlog is a list of features or technical tasks which the team
maintains and which, at a given moment, are known to be necessary and
sufficient to complete a project or a release
There are three different backlog types being them the following:
Product backlog
According to Scrum in the Wikipedia it's definition is:
The product backlog comprises an ordered list of requirements that a
scrum team maintains for a product. It consists of features, bug
fixes, non-functional requirements, etc.—whatever must be done to
successfully deliver a viable product. The product owner orders the
product backlog items (PBIs) based on considerations such as risk,
business value, dependencies, and date needed.
Or in other words, all of the tasks needed to be done to finalize the product.
Sprint backlog
The sprint backlog is the list of work the development team must
address during the next sprint.
In Scrum (for example) you develop in sprints. Usually sprints have the duration of 2 weeks. And for each sprint you should pick a set of tasks from the backlog that you think you will conclude in the duration of the sprint.
Release Backlog
The goal of a given release is to deliver a subset of the product
backlog, known as the release backlog.
Usually you develop your product by releases. First you determine which tasks go to each release, so you can release earlier versions of the product for the client to test and suggest modifications, without too much time and resources spent.
With that in mind, your To Do should be probably interpreted as the sprint backlog and your backlog the product backlog. Or you can use your To Do as a release backlog for instance if you don't develop by sprints.
In the column ToDo you should put the tasks that you want to work on within the current time segment. So you plan to get done with it within this segment.
All the rest of your tasks, you would work on in another time segment. This is my understanding about agile working processes.
If you don't work with time segments and just take the next important one when your current task is done, you should put the tasks there in the backlog, that are not ready enough to work on. For example if there are open questions to answer before starting with it.
I hope, this will help you.
Related
I'll start with the scenario I am most interested in:
We have multiple devices (2 - 10) which all need to know about
a growing set of data (thousands to hundreds of thousands of small chunks,
say 100 - 1000 bytes each).
Data can be generated on any device and we
want every device to be able to get all the data (edit: ..eventually. devices are not connected and/or online all the time, but they synchronize now and then) No data needs
to be deleted or modified.
There are of course a few naive approaches to handle this, but I think
they all have some major drawbacks. Naively sending everything I
have to everyone else will lead to poor performance with lots of old data
being sent again and again. Sending an inventory first and then letting
other devices request what they are missing won't do much good for small
data. So maybe having each device remember when and who they talked to
could be a worthwhile tradeoff? As long as the number of partners
is relatively small saving the date of our last sync does not use that much
space, but it should be easy to just send what has been added since then.
But that's all just conjecture.
This could be a very broad
topic and I am also interested in the problem as a whole: (Decentralized) version control probably does something similar
to what I want, as does a piece of
software syncing photos from a users smart phone, tablet and camera to an online
storage, and so on.
Somehow they're all different though, and there are many factors like data size, bandwith, consistency requirements, processing power or how many devices have aggregated new data between syncs, to keep in mind, so what is the theory about this?
Where do I have to look to find
papers and such about what works and what doesn't, or is each case just so much
different from all the others that there are no good all round solutions?
Clarification: I'm not looking for ready made software solutions/products. It's more like the question what search algorithm to use to find paths in a graph. Computer science books will probably tell you it depends on the features of the graph (directed? weighted? hypergraph? euclidian?) or whether you will eventually need every possible path or just a few. There are different algorithms for whatever you need. I also considered posting this question on https://cs.stackexchange.com/.
In your situation, I would investigate a messaging service that implements the AMQP standard such as RabbitMQ or OpenAMQ, each time a new chunk is emitted, it should be sent to the AMQP broker which will broadcast it to all devices queues. Then the message may be pushed to the consumers or pulled from the queue.
You can also consider Kafka for data streaming from several producers to several consumers. Other possibility is ZeroMQ. It depends on your specific needs
Have you considered using Amazon Simple notification service to solve this problem?
You can create a topic for each group of device you want to keep in sync. Whenever there is an update in dataset, the device can publish to the topic which in turn will be pushed to all devices using SNS.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I want to write a server program which accepts incoming connections and process received requests. The first idea appeared in my head was to use non-blocking socket with epoll() or select().
For instance, when epoll() returns, it gives me a socket array with available IO events. Then I have to loop over the socket array to send and receive data, once the buffer is entirely received or sent, a callback functions is executed. This is also the technique the most discussed on internet.
However, if I use this technique, my program will keep all the other sockets waiting while it is dealing with one client connection. Isn't it an inefficient way to go if the request of client is time-consuming?
In the documents that I've found they said that such one thread/process design can easily handle hundreds of connections simultaneously, and multithread design is always heavily criticized for its complexity, system overhead etc.
Thus, my question is: How to design an efficient server program if it has to handle heavy workload?
Thanks.
Million dollar questions with a million different trade offs. For those that get Monty Python...
https://www.youtube.com/watch?v=pWS8Mg-JWSg
Back to reality... Apache2 can handle heavy workloads, nginx can handle heavy workloads, so can Node, Tomcat, Jetty, JBoss, Netty... In fact any of the well knowns applications servers in use today and quite a few less well known ones can handle heavy workloads and they all use various combinations of threads, events and processes to do it. Some languages ie Erlang or Go etc allow you to easily spin up high performance application servers in a few hundred lines of code.
Although out of date now the following page has some great information on why this is not a simple problem...
http://www.kegel.com/c10k.html
Rather than worrying about performance now get something working, benchmark it then ask how to make it faster... if you've been smart and made sure that you have a modular design swapping out parts of it will be relatively easy ie look at what Apache did with MPM, a pluggable engine with completely different performance characteristics etc.
Once you have your server outperforming any of the above in benchmarks your answer to this question would likely be accepted.
Heavy workload is a misleading term and in the end, it doesn't really dictate how you should design your system. The main issue here, is one of responsiveness and its requirements. If processing a single request takes a long time, and you do not want to starve the other clients (which you probably don't), then a single thread design will obviously not do. You should at least have a thread (or one per client) that handles responding to the request in some manner, even if only to notify the client that the request is being processed.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have been doing r&d for the scope of SQL Server Service Broker to replace current messaging solution MSMQ. I want to know disadvantages of SQL Server Service Broker in contrast to MSMQ for following criteria.
Development
Troubleshooting
Performance (let's say we need to process 100,000 messages daily, having avg size around 25 KB)
Scalability
I've used Service Broker in my current project, having previously used MSMQ (brokered by MassTransit) with great success. I was initially dubious about using Service Broker, but I have to admit it has performed very well.
If you're using the Publish / Subscribe model, then I would use Message Queueing every time (though I would use RabbitMQ over MSMQ if the project allowed) but when you just want to chew through a stack of data and persist it to Sql Server, then Service Broker is a great solution: the fact it's so 'close to the metal' is a big advantage.
Development
Service Broker requires a lot of boilerplate, which is a pain, but unless you're planning on having lots of different queues it's manageable. Sql Server projects in Visual Studio take a lot of the pain of deploying away.
Troubleshooting
Service Broker is a black box - messages go in, and they usually come out, but if they don't then troubleshooting can be problematic, and all you can do is query the system views - and sometimes you simply can't find out what has gone wrong. This is annoying, but MSMQ has the same kind of issues..
Performance
Service Broker performance is excellent for our use case (see comment section below for discussion). We are processing a lot more than 100,000 messages per day, more than 30,000 per hour at our SLA load, and our message sizes are large. I would estimate we process close to 100,000 messages per hour during heavy load testing.
For best performance I would advise you to use a Dialog Pool like this one 1 as creating a Service Broker dialog can be an expensive operation.
You will also want to use the Error Handling procedures detailed by Remus Rusanu. (If you do use Service broker, you might as well read everything Remus has written on the subject before you start, as you'll end out reading it eventually!)
Scalability
You can certainly use more than one server to scale up if required, though we haven't had to do so, and from the load size you mention I don't think you would need to either.
I don't think I have really managed to answer your question, as I haven't highlighted enough disadvantages of Service Broker queues. I would say the impenetrable nature of its internal workings is the thing that most annoys me - when it works, it works very well, but when it stops working it can be very hard to figure out why. Also, if you have a lot of messages in a queue, using ALTER QUEUE takes a very long time to complete.
Not knowing how you are using MSMQ also makes it different to fairly compare the two technologies.
1 Recreated in a gist as the original url is now "disabled" and the page isn't in the internet archive. Eventually found a copy here
Would it be possible to build a Ticketmaster style ticket reservation system by storing all information in a Cassandra cluster?
The system needs to be able to
1. Display the correct number of tickets available at one time
2. Temporarily reserve a ticket while the customer is making the purchase
3. No two users can ever buy the same ticket.
For consistency all reads and writes should be made at quorum. I'm not sure how to implement steps 2 or 3?
Yes, you can.
However, there will be some transactions where you want strict consistency. For example, consistency does not matter when the user is browsing the site and adding tickets to their shopping cart, but when they checkout and select a specific seat number on a specific day consistency matters a great deal (double bookings being a bad thing, especially for high interest events).
So, you could implement 99% of the functionality in an eventually consistent database and implement the checkout process in a consistent database. This is also nice because you can scale 99% of your system that likely gets >70% of the load horizontally and across multiple data centers. Just keep in mind that you will have to deal with the scenario of your site being up but your checkout process being down (ex., an error dialog at checkout asking them to wait/retry and giving them a promo code for their troubles).
The last detail is that you will need to update your eventually consistent database's "number of available tickets" after someone checks out. The good news is that this can be done lazily - queue up that job and do it whenever your system has some spare cycles. It certainly never has to happen in the critical path of the user's checkout process.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
We have a small scrum-team which develops a webseite with a lot of users. We would like to improve our development and relase/deployment process, but there are a lot of concepts out there, which in our eyes don't fit together perfectly. Since we're not the only company developing a website, I thought it might be a good idea to ask here ;)
First of all, the software is not in good shape, so things like continuous delivery or continuous depoloyment are not possible because of bad test coverage and that's nothing we can change soon.
We have two-week sprints, so we develop new features or resolve bugs in that period, after the final sprint meeting we merge the feature branches into master (we use feature branches in git and pull-request for review and merging), do some testing, and deploy master to a public beta enveriment. There we usually find some bugs, resolve them and after that we deploy master inckuding beta fixes to the production enviroment. After that we start with our next sprint.
But that process is far from being perfect. First of all it's difficult to show features with all those branches in our sprint review meeting, and because we only have one master branch which gets deployed, we can't easyly deploy hotfixes to different enviroments. Sometimes we need more time for testing in our beta enviroment, so we can't merge new features to master, or same for production, once deployed, we can't deploy hotfixes if we are already testing new features on beta. If we expect bugs in beta or production due to larger changes, development has too keep feature branches for a longer period, so merging feature branches later on becomes more and more painful.
First we thought about long running branches, like master for development, production and beta. So we could merge whatever feature we want into one of the thre branches. But we really like to work with pull requests (reviews, feedback and deleting feature branch after merging), it's really nice to work with, but we can only apply a pull-request branch to one other branch. So here, we could merge to master without deleting the branch, and have to swithc to another tool to merge a feature to beta or production, or create new pull requests for beta and production. It works that way, but its not such a nice workflow as only merging to one master branch.
We also thought about git-flow (Vincent Driessen's branching model), which looks nice, but it seems to be better suited to software with traditional release cycles and versioning, not 100% for web aplications, which have no real versions but deploy everything which is ready after a sprint. It solves the hotfix problems, has an extra develop branch, but it requirs release-versions. So we could create a release branch, get problems sorted out, release it to production and delete the release branch. We could open a pull request for merging into master with the release branch, but problems start if we want to release it to beta (we use capistrano for deployment), because the branch changes every sprint. And what if we want to test features in our beta enviroment? We could use the release branch for beta, but this way a release to production has to wait until all features in the release/beta branch are ready. If testing of large features takes long, we can't upload small updates to production.
Next problem is versioning. We use jira and jira likes to use versions for releases. We could use versions like "1,2,3"..., but should one sprint be one version? Doesn't feel right, because sprint planning is not the same as release planning, or should it be the same when developing a web application? Because sometimes we develop featurs in a sprint, which take longer and are released later after being completed in the next 1-2 sprints. With git-flow, this changes could not be merged to develop until they are ready, because every release is branched from develop branch. So this way, some feature branches are not merged for a long time and merging them becomes more and more difficult. It's the opposite of continuous integration.
And last but not least, the deployment and QA process does not fit to scrum so very much, we don't have an own web operations team, when a sprint is ready, the product owner has to review the stories, we have to test them, deploy them to beta/production, and have to fix the problems in it immediately, which also interrupts or next sprint. We are not sure when is the right time to merge the feature branches? Before review meeting? This way we could have merged features which are not accepted by the product owner into the branch which should be released soon, but if we don't merge them, we have to demonstrate each feature in it's own branch and merge every accepted feaure branch, so integration and testing could only start after the sprint review meeting. Integration and testing could take days and needs development resources, so when should the next sprint start? After release to production? But this way we could not start a sprint every two weeks, and not every developer is needed for integrating and QA, so what should they work on? Currently we start the planning meeting for the next sprint immidiatly after the review meeting of the last sprint, but if we do it this way we are not sure how many time we need for the release which draws resources from the sprint...
So how do you release web applications?
What workflow do you use?
What's the best way if we want to integrate pull requests in the workflow?
How do you integrate QA and deployment into the scrum workflow?
I try to make a backlog and prioritize it:
Prio 1: Refit you Definition of Done.
I think you are betraying yourself how many features you can develop in a sprint.
Scrums defines the result of an sprint as "useful increment of software".
What you do is to produce a "testable increment of software".
You have to include the test into the sprint. All stories that are not tested and have a "production ready" stamp on it, are simply not DONE.
Prio 1: Do your retrospectives
Get the team together and talk about this release mess. You need a lightweight way to test stories. You and your team knows best what's available and what is hindering you.
If you already doing those retrospectives, this one is done. ;)
Prio 1: Automated Tests
How can you get any story done without UnitTests?
Start making them. I would propose that you search for the most delicate/important component in the system and bring test coverage (ECLemma/JaCoCo helps) up to at least 60% for this component.
Commit a whole sprint to this. I don't know your DOD, but this is work that has been procrastinated and has to be redone.
If a manager tells you its impossible, remind him of the good old waterfall days, where 2 weeks of development have not raised his eyebrow, why should it now...
Prio 2: Integration testing
You have Unit-Tests in place and you do some additional manual testing in the sprints.
What you want is to include the integration test (merging with master) to the sprint.
This has to happen, because you have to(!) solve the bugs in the sprint they have been introduced. So a early integration has to happen.
To get there it is important that during a sprint you work by priorities: The highest prio task/Story has to be solved by the whole team first. Goal is, to have it coded and unit tested, then you hand it over to your tester for smoke testing. Bugs found have the prio of the story. When it works good enough this part is deployed to the integration and tested there.
(Of course most of the time, not all team members can work effectively on 1 story. So one other story is processed in parallel. But whenever something stops your Prio1 Story, the whole team has to help there.)
Prio 2: Reduce new features increase Quality
To include more quality you have to lower expectations regarding amount of new features. Work harder is not what scrum tells you. It tells you to deliver more quality because this will speed up your development in mid term.
Regarding QA and deployment, I suggest the following:
Commit your code early to the main branch, and commit often. I am totally unfamiliar with git so I can't comment on how you're using it but I've always worked off the main trunk (using Subversion) with all the developers committing new code to the trunk. When we create a release we branch the trunk to create a release branch.
If you haven't done so already then set up a continuous integration system (we use TeamCity, but it'll depend to some degree on your stack). Get what unit tests you do have running at each commit.
Create a definition of 'done' and use it. This will stop you from being in the situation of testing features implemented during the previous sprint. Make sure your definition of done includes the passing of acceptance tests that have been co-written by your Product Owner, in my opinion the PO should not be rejecting features during the Sprint Review.