One of the major challenges we faced with the AllReady app was building a custom kudu deployment script. There is incredible power in this system, but it takes a bit of research, and a bit of help to get all the pieces working.
Let’s start with the simple goal: to make testing easier, we wanted to deploy to a live site automatically whenever we merged a pull request into master.
Azure supports this by default, including for asp.net 5 sites. Using Kudu was an obvious choice.
Life got complicated when we added a web job to the project. We added a web job because one of the actions the Allreedy application performs is to send messages to all registered volunteers. We don’t want to tie up a thread running in the asp.net worker process for the duration of sending messages through a third party service. That could take quite some time.
Instead, we want to queue up the emails on a separate webjob so that the site remains responsive.
That’s where deployment got complicated. You see, when we started, webjobs aren’t supported under asp.net 5 yet. The webjob builds using asp.net 4.6, using Visual Studio 2015. We also have one interface assembly that contains the types that are shared between the web application (asp.net 5) and the web job (asp.net 4.6).
So our build now includes:
1. Build the assemblies that are part of the web application using DNX.
2. Build the assemblies that are part of the web job using MSBuild. (Note that this means building one assembly twice)
3. Deploy the web site.
4. Deploy the web job.
Those extra steps require creating a custom deployment script.
Here are the steps we needed for adding our own custom Kudu script. There were several resources that helped create this. First, this page explains the process in general. It is a shortened version of this blog series (link is to part 1).
The first task was to create a custom build script that performed exactly the same process that the standard build script process performs. I downloaded the azure tools for deployment, and generated a deployment.cmd that mirrored the standard process.
You need to have node installed so you can run the azure-cli tool. Then, install the azure-cli:
npm install azure-cli –g
Then, run the azure CLI to generate the script. In my case, that was:
azure site deploymentscript –aspWAP allreadyApp/Web-App/AllReady/AllReady.xproj –s allready.sln
Notice that I’m directing azure cli to generate a script based on my xproj file. But, notice that this does not build the .csproj for the web jobs.
Before modifying the script, I wanted to verify that the generated script worked. It’s a good thing I did, because the default generated script did not work right away. The script generator assumes that the root of your github repository is the directory where your .sln file lives. That’s not true for allready. We have a docs directory, and a code directory under the root of the repo.
So, the first change I had to make is to modify the script to find the solution file in the sub-directory. After that, the script worked to deploy the website (but not the webjobs). Doing that build required three changes. First, I needed to restore all the NuGet packages for the .csproj style projects:
call :ExecuteCmd nuget restore "%DEPLOYMENT_SOURCE%\NotificationsProcessor\packages.config" -SolutionDirectory "%DEPLOYMENT_SOURCE%" -source https://www.nuget.org/api/v2/
Getting this right got me stuck for a long time. In fact, I needed to get some product team support during our coding event from David Fowler. The version of nuget running in Azure needs to use the V2 feed when it’s restoring packages for a .csproj based project. *Huge* thanks to David for helping us find that.
Next, we needed to build the .csproj bsed projects:
call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\AllReady.Models\AllReady.Models.csproj"
IF !ERRORLEVEL! NEQ 0 goto error
call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\NotificationsProcessor\NotificationsProcessor.csproj"
IF !ERRORLEVEL! NEQ 0 goto error
The final step is to deploy the webjobs. This meant copying the web jobs into the correct location. This step happens after the build, and before the Kudu Sync process:
call xcopy /S "%DEPLOYMENT_SOURCE%\NotificationsProcessor\bin\debug" "%DEPLOYMENT_TEMP%\wwwroot\app_data\jobs\continuous\notificationsprocessor\"
The deployment script and the .deployment config are in our repository, so if you want to explore, check it out. Our repository is here: http://www.github.com/htbox/allready. And, if you want to help, check out the issues and send us a pull request.
Last weekend, Humanitarian Toolbox held a very successful coding event at That Conference. We got quite a bit done on both the crisis checkin and the All Ready applications. Thanks to the organizers of ThatConference, and everyone that attended during the weekend. I’m always impressed by how many developers join us there, and by how much they contribute.
OK, enough of the public service announcement. I wrote this post to help with a common issue: Pull Requests vs. Commit Privileges.
All the Humanitarian Toolbox projects use the Fork & Pull model for development. It enables us to keep the core contributor teams small, and yet enables anyone that wants to contribute to make changes and submit them. I, sadly, didn’t announce that clearly enough to all the volunteers when we started our event. Many of the volunteers thought we were using the Shared Repository Model.
That meant that later in the event, I had a number of developers come to me with issues because they could not commit to the main repository. That’s because they didn’t have the rights.
Announcement: Don’t fix this by just giving commit privileges. It’s not necessary.
Announcement II: Don’t fix this by trying to copy your changes and merging by hand. It’s not necessary either.
Thankfully, this is really easy to fix, once you understand how git works. It does also require using the git command line. The git command line isn’t that hard, and it’s important to learn as you do more with git. If you ever run into this issue, follow the instructions below to fix it.
Before I explain the steps to fix the problem, let me describe what happened, and why it’s a problem. Look at the image below. It’s a portion of the main window in my Github for Windows application. You can see that I have two copies of Crisis Checkin on my local machine. The top one is the clone of a fork that I made in my account. The bottom one is the clone of the master HTBox repository. These two copies have different remote repositories.
If I run ‘git remote –v’ in both directories, you can see the difference. Here’s the output from my fork:
C:\Users\billw\Documents\GitHub\TheBillWagner\crisischeckin [master]> git remote -v
origin https://github.com/BillWagner/crisischeckin.git (fetch)
origin https://github.com/BillWagner/crisischeckin.git (push)
Note the difference when I run the same command in the main repository:
When you execute a ‘git push’, you’ll send your changes to the git repository identified by origin (by default). If you cloned the main repository, git will try to push to that main repository. If you forked, and then cloned your fork, git will try to push to that forked repository.
The goal is to push the changes you made in your desktop to a fork of the repository where you have commit rights. The drawing below shows the relationship between the repositories, and the commands used to create each one.
To save your work, and create a pull request, you’ll need to create a fork, and push the work you’ve already done to that fork. The specific commands are as follows:
I usually create a fork from the github.com website. (Just press the “fork” button in the upper right corner). That creates a copy of the main repository in your account. This copy is also located on the github servers, not on your drive.
This fork is where you want to push your changes.
Now, you need to configure your fork as a remote repository for your working directory. Looking at the image above, you want to push from your desktop (where you have made changes) to the fork (the upper right repository).
You add a remote by typing:
‘git remote add fork https://github.com/BillWagner/crisischeckin.git’
Replace ‘fork’ and the URL with a name you want to use and the URL of your fork. I use ‘fork’ as the name, because it’s easy for me to remember.
You can add new remotes as long as the additional remotes are related to the origin remote. (Unless you start forcing git to do things. That means you can’t accidentally push your changes to crisis checkin to a fork of the Roslyn project (for example).
Now, your local copy is configured with two remotes: The source for the application (owned by HTBox) and your fork (owned by you). These two remotes are named ‘origin’ and ‘fork’.
Now, you need to push your changes from your working area to your fork. That’s just a git push, and specify the fork as the remote:
‘git push fork’.
By adding the extra parameter to the push command, you specify the remote repository where you changes should go.
It’s that easy.
Unless it isn’t. If it has been a while since you cloned the original repository, you may need to sync your working directory with upstream changes. That’s documented here. It’s variation of what I just described, but you should be able to follow it.
After you’ve pushed your changes to your fork, you can open a pull request to discuss your changes and get them merged into the main repository.
After I’ve finished this work, I will often make a new clone of my fork for a given repository. That way, the default remote (referred to by ‘origin’) points to my fork, rather than the main project.
I hope that helps.
This past Monday, Microsoft released the production version of Visual Studio 2015. Let’s get right to the lead: Visual Studio Community edition is free (for independent and Open Source developers). You have no excuse not to get this.
I’m not going to repeat all the information about the release and the total set of new features. Soma did that on his blog quite well. Instead, I’m going to focus on my areas of interest, and some of the resources I’ve written that can help you learn about these new features.
There are a number of new features in C# 6. Members of the team have been updating this page on Github with information about the new features.
I’ve written a number of articles about C# 6 for InformIT. The first four are live:
I recorded a Microsoft Virtual Academy Jump Start on C# 6 with Anthony D. Green, one of the program managers on the Managed Languages team.
And finally, I’ve written quite a few blog entries on the new language features. You can see a list here.
This version is particularly exciting because of the new compiler services that are part of Roslyn. These are a rich set of APIs that enable you (yes, you!) to write code that analyzes C# code, up to and including providing fixes for mistakes or other poor practices.
I’ve written an article for InformIT about the analyzers. You can read it here. I also did a Jump Start for Microsoft Virtual Academy with Jennifer Marsman. If you missed the live event, watch the Microsoft Virtual Academy home page for updates. The recording should go live soon.
You can also learn more by exploring some of the open source projects that contain analyzers and code fixes:
Finally, I was thrilled at the contribution from members of the Visual Studio team to Humanitarian Toolbox. Several members from different parts of the Visual Studio team worked for three days to build the initial release of the allReady application. The application source is on Github, under the HTBox organization.
This application was requested by the Red Cross. It provides features to lessen the impact of disasters on familes and communities. You can learn more about the project here on the Humanitarian Toolbox website.
The Visual Studio 2015 launch event included a profile of the developers and the process for building the initial code for allReady. You can watch the entire event on Channel 9. If you are interested in just the allReady app, and how it was built with Visual Studio 2015, look for the “In the Code” segments. There may be more In the Code episodes coming as the application grows.
All of us at Humanitarian Toolbox are grateful for the contribution from the Visual Studio team.
As a developer, I’m also grateful for the great new tools.
We are back from ThatConference after hosting another successful codeathon. I'm always thrilled to see the participation we get from our friends in the upper midwest. A couple dozen developers gave up their entire weekend to write code for Humanitarian Toolbox on the Saturday and Sunday before ThatConference.
I also must thank Falafel Software for their generous sponsorship that enabled us to host the event again. We greatly appreciate the support.
We worked on two different applications while there: Crisis Checkin and the Humanitarian Data Bus.
We spent two days working on the mobile applications for Crisis Checkin, using the Xamarin toolkit.We had a UX designer at the event, so we made the most of the opportunity. At the end of the two days, we now have UI screens for all the major interactions on all the major mobile platforms. We have an API design for the server side code, and we're ready to start implementing the WebAPI services.
The Humanitarian Data Bus is a new project. The purpose of this application is to provide a common interchange mechanism for disaster relief organizations. There's a large amount of data that different organizations collect during recovery and rebuild. This application will enable those different organizations to share that data. It will mean that everyone involved has better information, and therefore makes better decisions during the recovery and rebuild operations.
The overall architecture was created, and the initial shell of the application was coded. It's had a strong kick start, and we'll be getting that code on our github page soon. When it appears, we'll be ready for more work and more pull requests.
Again, I want to thank our hosts at ThatConference. They did a wonderful job to make our codeathon a success. They provided space, food, and got the word out to interested developers. We couldn't have done it without your support.
I’m glad that I was selected to speak at self.conference, coming to Detroit this May.
The speakers and sessions are a strong mix of the technical community in this area. In addition to quite a few local speakers, other super smart folks from the midwest are apeaking.
I also want to point out Tony Surma’s talk on “Apps that work when nothing else does.” Tony is a fellow board member for Humanitarian Toolbox, a charitable organization that creates Open Source applications for disaster relief. I’m the president and a board member as well. We’d love to have your help developing these applications. And, as you’ll learn from Tony’s talk, there’s a lot of software product skills you’ll learn developing apps that have to run in a disaster zone.
Self.Conference promises to be a great, great event. Go to http://selfconference.org/ and register today. See you in May.
Well it's been heaven, but even the rainbows will end.
Now my sails are fillin' and the wind is willin'.
And I'm as good as gone again.
I'm still walkin', so I'm sure that I can dance.
Just a Saint of Circumstance, just a tiger in a trance.
- John Perry Barlow
My last post was looking back, this one looks forward. I am truly excited about how I spend my time now.
I'm concentrating my time on the activities I enjoy the most: creating software that helps people, and helping developers become better at what we do. There's four different activities that are part of this plan.
First, I'm working with a set of awesome colleagues to launch The Humanitarian Toolbox. We are working with the global developer community to create, deploy, maintain and support software applications that aid humanitarian relief efforts. These are especially important in times of crisis. I've been doing volunteer work with them since February. I've been learning about current disaster response practices, and how software can enable relief workers to spend more time on what's really crucial: helping the people affected by the disaster. We're working with global relief organizations to identify needs. We're scheduling events to launch teams on the most pressing needs. We're working on the deployment and maintenance plans. I'm truly inspired to work with such a great group of dedicated people with one simple goal: Save lives and reduce the impact of disasters.
Second, I'm creating courses for Pluralsight. I'm excited to be part of Pluralsight. I've spent my entire career developing software and teaching software development to others. Working with Pluralsight gives me a new way to reach developers world wide and help them become better software developers. But what's more exciting to me is the community of Pluralsight authors. The list of Pluralsight authors contains people I've known and respected for much of my career. It also contains many of the smartest, most energetic rising stars in our industry. I get incredible energy from this group. It's the most gifted, intelligent and hard-working community I've ever been a part of. Every morning I log into the Pluralsight Yammer network, and join a group of great minds. There are amazing conversations about everything related to software development. If you're not already a Pluralsight subscriber, you should be. It's a great way to learn from our industry's best minds.
Third, you'll see me at more conferences. Both activities (Humanitarian Toolbox and Pluralsight courses) enable me to attend and speak at more conferences. At many of them, I'll be running Humanitarian Toolbox hackathons. At many, I'll be speaking on developer topics I believe that will help you be a better developer.
Fourth, I'm not done writing books. I think books and online training are both valuable resources to learn software development. And, I enjoy creating both. I haven't signed a book contract at the moment, but I'm working on a couple outlines for topics.
I'm excited about the wealth of opportunities ahead. I'm going to spend more time on technology, and on helping other developers improve their skills. I get to spend almost all of my work time on the activities I'm most passionate about. I'll continue blogging here regularly, and I'm inviting my readers to join the conversation. What do you want to learn? Are you interested in helping humanitarian toolbox create software? Let me know in the comments.
ThatConference is just around the corner. There are two parts of the event where I'll have a big presence.
First, The ThatConference GiveCamp. I'm one of the project leads, for the volunteer checkin system. Visit the link above and you'll get a great overview on what we are building. The GiveCamp / Hackathon takes place on Saturday and Sunday. Come join us, and build some code that can have a real impact on people. We can make humanitarian response more efficient, and literally save lives. The other project being developed at the ThatConference GiveCamp is the Mobile Based Emergency Training project that Humanitarian Toolbox started at DevTeach. Phil Japikse will be at ThatConference and leading that project again.
Also, a big thanks to Telerik for providing Humanitarian Toolbox with copies of their tools for the projects we are creating.
I'm also excited to be speaking there this year. My session will discuss the reasons to minimize the use of async void methods. Come join me on Monday afternoon and get your async on.
Finally, I'm going to be doing more work for Humanitarian Toolbox between other sessions. The main focus of Humanitarian Toolbox is to own the projects over time. We want them ready to deploy when disaster strikes. I'll be hanging out, adding more code to the crisis checkin tool. Please join me. I'll post on twitter where I'm coding.
If you're coming to ThatConference, join us.
I’m a bit late getting this up. It’s been one of those months. I’m attending and speaking at my first ThatConference this summer.
My talk is on the dangers and misuses of void returning async methods in C# 5. I’ll discuss the problems with that practice, and go over some of the practices that can be used to avoid that practice. It’ a wicked strong lineup of speakers and content, and I do hope you can make time to attend my talk.
I’m also going to be running a hackathon/GiveCamp for the Humanitarian Toolbox. Right now, we’re working with disaster relief experts to pick the right project. If you want your software to make a difference the next time there’s a natural disaster, like the tornados in Oklahoma this past week, come to ThatConference the 10th and 11th to participate. Write some code. Change the world.
The Humanitarian Toolbox is hosting its next Hackathon at DevTeach in Toronto.
DevTeach is a fantastic community-led conference that is celebrating its 10th anniversary this year. Many of my friends in the MVP and RD communities are speaking there. It’s a fantastic way to learn more about software development, and grow your skills.
This year, you can give back as well.
You can spend one evening (May 30th) using your skills to build software that makes relief workers more efficient when disaster strikes. Attend the Humanitarian Toolbox hackathon at DevTeach. Help build the next generation of life-saving tools to meet the needs of relief workers. Your skills can save lives.
We held our first hackathon at DevIntersections in April. (You can read about the experience on my previous blog entries). We’re hoping to build on that momentum during a number of conferences this summer. DevTeach is our first chance to grow more. Richard Campbell (yes, that Richard Campbell) will be hosting this hackathon. If you are going to DevTeach, please join us.
scheduling note: Due to family events (happy ones), I can’t make this conference. I’ll be at other upcoming events later this summer and fall.
I've always found that developers are very interested in contributing their skills and their time to help others.
I'm excited to be part of a new opportunity for developers to help others: The Humanitarian Toolbox.
The Humanitarian Toolbox is the brain child of Richard Campbell, of .NET Rocks fame. The concept is to have developers create Open Source software projects that solve real problems for disaster relief organizations. A group of people, including Richard and I, have been working with global relief organizations to determine what they need most. We've already got a list of more than a dozen application ideas.
It's time to start building. We've picked the first project: a relief worker check-in system. This system would enable relief workers and volunteers to check-in and check-out at a disaster site. The field coordinators would have better information about the skills and the availability of relief workers, enabling them to better deploy the people that have volunteered their time to help. Relief workers and volunteers would get assignments and locations directly on their smartphone. The end result will be that relief workers are more efficient while on site, and more lives are saved.
We need developers that want to build this app.
Humanitarian Toolbox is hosting a hack-a-thon at DevIntersections. I'm working to organize the event, and help organize developers (like you) interested in participating. We plan to create a small proof of concept for the relief worker check-in system during the hack-a-thon. After the initial hack-a-thon, we want to enlist the broader community to continue building the app.
What's our deadline? We'd like it in place before the next natural disaster happens. We don't know when that will be, so we'd like to keep building and have it ready as soon as we can.
If you're going to be a DevIntersections, join us. Contribute your talent and help relief workers save lives. If you're not going to DevIntersections, there's still plenty to do to help. Follow us on twitter: @htbox or the hash tag for the event: #HtBox. We'll have plenty more software to build after the hack-a-thon. For us to be successful, we'll need to keep building and enhancing software for relief workers to battle the next crisis.
Write some code, help the world. Contribute to Humanitarian Toolbox.
All of these projects are Open Source (using the Creative Commons license for content, and the MIT license for code). If you would like to contribute, visit our GitHub Repository. Or, if you have questions, comments, or ideas for improvement, please create an issue for us.