AF CyberWorx is focusing on Human-Centered Design for the month of October. It’s the secret ingredient of all that we undertake and can be an extremely valuable addition to most processes. Please follow us on social media to gain more insight into the value of Human-Centered Design and enjoy this week’s blog post by our lead UX/UI designer, Mr. Larry Marine!


A recurring sprinkler control problem for large spaces (parks, ball fields, etc.) involves adjusting the sprinkler schedule to accommodate special events. A 3-day baseball tournament requires more than just shutting off the water for 3 days. Watering at night makes the field soft and susceptible to damage, so that’s not an option, either.

A more successful approach involves a complex schedule of watering more than usual for 2 days prior to the event, sprinkling only a few minutes each night of the tournament, then watering heavily again after the tournament. Existing solutions require users to manually change the watering schedule each day in the complex irrigation control UI, which almost always results in a user error.

A knowledge-based approach would be to provide different watering templates for various types of events that could be invoked by the users. For instance, the user might merely indicate that a 3-day tournament will occur on such-and-such dates and then the system would automatically adjust the watering cycles before, during, and after the tournament, returning to the standard settings after the event. Moreover, the templates could be based on best practices gleaned from other users with similar field composition and irrigation equipment.

By providing a design that asks the user to indicate the event and the dates, we can provide cues that focus on the knowledge we can expect users to actually have. So rather than a complex set of features, the user only needs to select the event and the dates, and the system does the rest of the work.

As technology becomes ever more complex and pervasive, we cannot expect users to understand how to use every device with complete expertise. They may be familiar with some devices, but not all. Therefore, we must design device interfaces so that users can use them without having expertise for that device.

A common approach to UI design is to provide a cornucopia of features and expecting users to know which features to use and when to use them. Unfortunately, this is a failed premise evidenced by the frequency of user errors. This is even more true when users are distracted by other factors such as state of mind and physical limitations. For instance, a person can’t be expected to attend to a complex device with patience and clarity when faced with a panic situation or if they are temporarily impaired.

Beyond simple UX Design, knowledge design builds knowledge into a design to help the average user bridge existing levels of skill and knowledge to be more successful. The key is to identify the knowledge users need in order to succeed and finding ways to embed that knowledge into a design.

Successful techniques for designing knowledge into a UI include designing best practice approaches into the product, creating task-oriented designs, providing templates and intelligent defaults. These techniques require that the designer have enough in-depth knowledge of the domain such that they can provide salient cues for the users.

CyberWorx uses techniques specifically focused on identifying knowledge design opportunities within a problem domain. We would be happy to discuss these techniques with your team. Just ask.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.


AF CyberWorx is focusing on Human-Centered Design for the month of October. It’s the secret ingredient of all that we undertake and can be an extremely valuable addition to most processes. Please follow us on social media to gain more insight into the value of Human-Centered Design and enjoy this week’s blog post by our lead UX/UI designer, Mr. Larry Marine!


Among the more popular forms of user research, usability testing is often conducted to gain insight into the users’ perceptions of a design. Like any other tool, usability testing yields the best results when performed correctly. An incorrectly performed usability test can lead you down the wrong path.

A common test design approach is to include specific directives on how to use a product, such as “print out a receipt.” While this seems innocuous enough, it actually biases the users’ behaviors and consequently the results. If users think they know the objective, they inherently try to please the tester. It’s in our nature. Users may even ‘game the system’ (focus more intently than normal on the stated outcome) to try and please us.

Rather than telling the users to do something, try using a question that can only be answered by performing a set of tasks. The advantages of using questions are plentiful:

  1. As humans, we love to solve problems or answer questions. Using a question in a test creates an intrinsic motivation that elicits a more realistic behavior. If users know what you are looking for, they are likely to alter their natural reactions and focus more attention on that specific action.
  2. A technique to hide your intentions about what you are testing for is to ask a question about an aspect of the task that occurs after the thing you are focusing on. For instance, instead of telling the user to print a receipt, ask them, “Can you tell how many pages are included in the printed receipt?” They will print a receipt as part of the question without realizing that is what you are focusing on.
  3. Questions reduce the tension a user might feel to perform a task. If they are given a directive, they believe it can be done. Thus, if they cannot complete the task, they feel it is their failure, not the design’s. If they cannot answer a question, they can legitimately say they don’t know the answer, which is less stressful and does not influence their behavior.

Another test design failure involves unintentionally biasing users by providing an incomplete prototype that lacks the screens users would run into when they veer off course. It’s not necessary to have a complete prototype, but at least have the screens a user would likely run into for a common set of errors for the tasks you are testing.

In the real world, users won’t immediately realize they have made a mistake and will continue to click around a few more screens. This observed behavior gives real insight into how the users finally determine that they have made an error and then how they try to get back to where they made the error.

A prototype that only has screens and interactions for the happy path unintentionally informs the users when they have taken a wrong turn, thus interrupting their natural behaviors. This limits your ability to identify what cues the users are relying on to determine if they are going down the right path and how they recognize when they haven’t. A key tenet of usability testing is that you learn more from watching users making and recovering from mistakes than you do from them NOT making mistakes.

Another common mistake is to rely on usability testing as the only user research method. Usability testing is an excellent method for capturing evaluative insights but is not that accurate for providing generative insights. Watching people use your design only informs you about the existing design, not about truly innovative design approaches or unmet needs. These require more generative methods such as interviews or observations.

Usability Testing can be a very useful tool, or it can mislead you if performed incorrectly. Biasing the users with a poorly conducted test yields inaccurate insights leading to design changes that solve the wrong problems. There is a major difference between what users say and do, and you should always lean towards performance rather than user preference. You must strive to design your test to collect accurate information, otherwise you are just wasting your time and your resources.

The team at AF CyberWorx can help you plan your next Usability Test (or any user research) to gain the maximum benefit.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.

Recent USAFA and AF CyberWorx Graduate Flourishes in New Position

AF CyberWorx Post Graduate Update Series

AF CyberWorx reached out to former cadets who worked with the innovation organization on campus before graduating. We caught up with two alumnae and found out what they are doing now, how their experience in AF CyberWorx influenced them, and what today’s cadets can do to better prepare for a successful future. Read our first alumna feature today.

2d Lt Emily Snyder, graduating class of 2018, and political science major, is currently a graduate student at the American University in Washington, D.C. She is studying international affairs with a focus on U.S. foreign policy and cyber security as it relates to national security. After finishing graduate school, she will become an Air Force cyber officer. While a student at USAFA, Snyder and her teammates worked together to develop a humanitarian application that would enable refugees to access information and form social networks while awaiting news of their asylum status in new countries with the goal of reducing refugee uncertainty.

Although the project was not completed before Emily graduated, she did learn a copious amount about the app development process. The AF CyberWorx project gave her the opportunity to think less in a traditional bureaucratic way, and more in a “Silicon Valley,” freeform mindset. The ability to think outside the box without penalty has immensely helped Emily with her current course load as a graduate student and as an intern at the Pentagon. While working on the project, her team collaborated with industry leaders at Pivotal Labs who elaborated on the creative process of application development. Those same connections she made at AF CyberWorx are still helping her today; Emily is working with Pivotal yet again during her internship and has been able to bridge the gap among her peers, thanks in part to the human-centered design she learned and applied in the project.

AF CyberWorx also opened Emily’s eyes to the importance of cyber security and how deep the field is. She hopes that in the future she can aid in outpacing policy with the implementation of new technology. She believes that giving the Air Force the ability to no longer need to play “catch-up” would create better mission succession rates.

Emily has a few words of wisdom for the third- and fourth-year cadets to keep in mind as they continue on their path of success within the Air Force Academy.

Be sure to focus on cyber. As the Air Force becomes more technically savvy, there will be an influx in the cyber field and positions that will need to be filled. Cyber will eventually become the new backbone of the Air Force, which is something you can gather by just seeing how much it has grown in the last 10 years. If you want to be successful within the Air Force’s cyber field, focus on what AF CyberWorx can teach you now while you’re in school, and you’ll be better prepared. On the same note of being focused on your future, be sure to pay closer attention to today’s current events…The problems you are seeing today will be the problems you need to solve in the future.

Do you have a story you’d like to share from your experience with AF CyberWorx? Tell us all about it, and you, too, could be featured on our blog. Email Jacki Stewart at

Want to stay updated on all things AF Cyberworx? Be sure to follow us on FacebookTwitterLinkedIn, and subscribe to our emails!

USAFA Cadets Win 1st Place at Cyber 9/12 Strategy Challenge

CyberWorx Cyber Win 9/12 Challenge

There is no rest for a U.S. Air Force Academy (USAFA) cadet. The USAFA Cyber Competition Team had the opportunity to visit Washington D.C. over spring break and participate in a cybersecurity breach challenge. They placed 1st within the student-based track and beat out 37 other competitor teams in the 7th Annual Cyber 9/12 Strategy Challenge

The Cyber 9/12 Strategy Challenge is an annual cyber policy and strategy competition where students from across the globe compete in developing policy recommendations tackling a fictional cyber catastrophe. It is a one-of-a-kind competition that helps prepare future policymakers to learn how to handle what to do the day after a “Cyber 9/11” or a “Digital Pearl Harbor.” Students gain a deeper understanding of the policy challenges associated with cyber crises and conflict during the part interactive learning, part competitive scenario exercise. 

The teams are challenged to respond to a realistic, evolving cyber attack while simultaneously analyzing the threat it poses to interests on national, international, and private sector levels. This year’s challenge was to confront a fictional cyber incident in three rounds, involving a nation-state’s attempt to degrade trust in the US 2020 Census. The students composed policy recommendations and justifications of their decision-making process while considering the role and implications for the relevant civilian, military, law enforcement, and private sector entities and updating their recommendations as the scenario evolves. 

Delogrand USAFA Cyber Competition TeamThe USAFA Cyber Competition Team, Delogrand, was comprised of four cadets First Class, James Lynch, James Brahm, Madison Tung, and Dylan Raess. The participating students had a unique opportunity to be coached by expert mentors and high-level cyber professionals. In this team’s case, they worked with Maj Timothy Goines (DFL), Capt Justin Raynor of Computer and Cyber Sciences (DFCS), and MSgt Christy Treasure of AF CyberWorx (DFK). Delogrand received the grand prize of paid flights and lodging to the 8th International Conference on Cyber Engagement on April 23 in Washington, DC. 

The cadets who attend the event have the opportunity to develop valuable skills in policy analysis and presentation while competing against teams from universities across the world at the 9/12 Cyber Challenge.

“This competition is different than most because not only do the cadets use the ‘hands-on’ ‘technical knowledge gained throughout the year, they get to expand their knowledge into the policy and strategy of cyber,” stated mentor MSgt Treasure. “This allows the cadets to deepen their understanding of how to legally use those ‘cyber effects’ within the context of cyber warfare and protecting the nation, which will be useful as they become leaders across our country.” 

The Academy also sent C1C Ryan Ramseyer, C1C Joanna Trevino, C2C Keith Billiot, C2C Bailey Compton, C2C Garret Gwozdz, C3C Thomas Galligani, C3C Benjamin Kazules, and C4C Grady Phillips as participants in the event. Congratulations to all of the USAFA cadets!

Cyber Competition Team AF CyberWorx Blog

Want to stay updated on all things AF Cyberworx? Be sure to follow us on FacebookTwitterLinkedIn, and subscribe to our emails!


Optimis April Email Newsletter AF CYberWorx


There is a pilot shortage in the United States Air Force right now and part of the problem affecting pilot retention is the current evaluation process. It can take up to three months for a student pilot’s evaluation to be submitted after their evaluated flying time. 

Read about the app that can help change all that in our recent blog.

Read Now!

Innovation April Email News 2019



Airmen, did you know that you can tell us what to work on next? Send your ideas to IdeaScale today and work with us by bringing the future faster for the United States Air Force. 

Submit Idea ?

Collaboration April Newsletter AF CyberWorx



AF CyberWorx, together with our partner C-TRAC, will be hosting a Technology Transfer event in June to highlight the technology and resources available at USAFA. Stay tuned for more details!

Space Symposium AF CyberWorx April News


Will you be in town for the Space Symposium? Come visit us at the Catalyst Campus Open House on Wednesday, 10 April, 1:00-4:00 pm in the Peak Technology Room. Catalyst Campus is located at 555 E. Pikes Peak Avenue, Colorado Springs.

Directions ?

C-TRAC Announces New “Collider” Series Of Free Technology Networking Events At The Catalyst Campus

C-TRAC Announces New “Collider” Series Of Free Technology Networking Events At The Catalyst Campus

The Public Is Invited to the Inaugural Collider Event for Snacks, Speakers and Socializing

C-TRAC, the Center for Technology, Research and Commercialization, is excited to announce the first in a series of Collider events – a new series of open houses intended as learning opportunities combined with a meet-and-greet for innovators, technologists, venture capitalists and anyone who might be interested to learn more about local innovations and start-up opportunities here in Colorado Springs.

The first Collider will be held on March 7, 2018, from 4:30 – 6:30 PM, at the Railyard on the Catalyst Campus, 555 E. Pikes Peak Ave., Colorado Springs. The focus will be on showcasing CyberWorx, a highly-innovative Air Force program that educates airmen while simultaneously partnering with industry to solve cyber problems facing our nation. Design thinking – a structured framework for understanding and pursuing innovation in ways that encourage outside-the-box thinking – is prominent in the CyberWorx process.

Lt Col Michael Chiaramonte, Director of Operations for CyberWorx at the Air Force Academy, will briefly address the attendees to discuss what CyberWorx does and how this benefits the Air Force and the technology community.  Lt Col Chiaramonte will be followed by Patty Bonvallet, Technology Development Manager of Boecore, Inc., a Colorado Springs woman-owned aerospace and defense engineering company specializing in cyber security, software solutions, system engineering, enterprise networks and mission operations. Ms. Bonvallet will discuss her experience at CyberWorx and how it has benefited her personally, as well as Boecore as a corporation.   A brief demonstration of the CyberWorx brainchild called the Command Risk Ecosystem, a cutting-edge cybersecurity technology, will also be presented.

After these short presentations, attendees can mingle with the speakers and the CyberWorx team to learn how to get involved in the cutting-edge CyberWorx program as industry mentors and partners. Some government leaders are also expected to attend, with the chance to chat with them. Light hors d’oeuvres, wine and beer will be served as attendees enjoy a wealth of opportunities to network with movers and shakers in industry and technology here in Colorado Springs.

This event is FREE and open to the public. Go to to let C-TRAC know you plan to attend.

CyberWorx Design Sprint Examines Multi-Domain Command and Control Challenges

CyberWorx Design Sprint Examines Multi-Domain Command and Control Challenges

COLORADO SPRINGS, Colorado, December 5, 2017 – C-TRAC, the Center for Technology, Research and Commercialization, today helped host the concluding out-brief for CyberWorx’ #AFMDC2 semester-long design project at the United States Air Force Academy. 

CyberWorx partners with industry to solve cyber problems facing our Air Force and nation while simultaneously educating Air Force cadets and Airmen. Design thinking – a structured framework for understanding and pursuing innovation in ways that encourage outside-the-box thinking – figures prominently in the CyberWorx Sprint process. Industry experts and thought-leaders are recruited to participate in design Sprints, acting as mentors during semester-long projects and holding the cadets to a very high intellectual and technical standard for their proposed solutions. 

#AFMDC2 is the acronym for Air Force Multi-Domain Command and Control, a priority capability currently being examined at several levels of the Air Force. It refers to the ongoing effort to make operational units more effective by improving the speed and quality of decisions by Air Force Commanders who are overseeing the command and control of multiple operational domains, such as Air, Space and Cyber. Automating decision support, ramping up human-computer teaming, and emphasizing mobility, technological simplicity, and highly-intuitive interfaces and processes are areas of particular focus for improvements. The goal of this CyberWorx project was to offer Air Force leadership viable options for simplifying the command and control of multiple domains while increasing their agility during integrated air, space, and cyber operations. 

After a semester of working on these issues, two groups of cadets with their industry coaches provided an out-brief to present the innovative solutions they developed. To an audience of Air Force experts, USAFA instructors, and other interested industry partners, the teams of cadets cleanly broke down the complex web of related issues surrounding their specific challenge – or problem statement – using real-life use cases and mission threads as sample scenarios. They described the comprehensive, unifying solution each team developed. 

One team sketched out a simple but multi-functional app to better manage equipment for Rescue Squadrons and similar units. Wiping away generations of bureaucratic tradition, the proposed app allows airmen to requisition equipment from a menu of items with a few clicks. The approval process would be drastically streamlined. Contractors would be able to bid on the equipment requests, including uploading proposal documents, and there would be more overall transparency to the process, including better oversight to prevent duplication of requests. The team postulated that the current multi-year turnaround on equipment requests would be shortened dramatically by the new app and simplified process, enabling warfighters to achieve multi-domain effects much more quickly.  

The other team focused on a very ambitious project underpinned by AI – artificial intelligence. They proposed a software solution to the complicated process of creating a “strike package” for a mission. A process that normally takes many precious hours not always available in wartime, these cadets envision a future where an AI algorithm with current mission requirements, past mission histories, available assets and munitions – even weather reports—will aid commanders in decision-making. Crunching this data in a matter of moments, the “Strike Package Automator” will present a viable strike package to the planner, who then evaluates and tweaks it without the many hours of stressful work currently involved in creating a strike package one capability piece at a time. 

When asked what surprised them most while researching this project, the cadets responded that there are a surprising number of antiquated, paper-based practices involved in strike planning, all of which have had technological solutions for years. The team expressed the conviction that their envisioned software package, when extended across all domains, would make a significant dent in many domain command and control issues currently clamoring for resolution. 

Air Force Colonel Max Lantz, one of the officers to whom the outbrief was directed, was complimentary of both teams’ ideas. At the outset, Col. Lantz noted, “I’m very honored to be here; I look forward to hearing the bright ideas they have.” After each presentation, compliments such as “Phenomenal job” and “Awesome” were offered up by the colonel who serves on the Air Force Space Command staff and works closely with the Air Force’s overall MDC2 team.  

The cadets presented themselves as mature and organized, producing some clearly actionable ideas that demonstrate the continuing advantages of using the CyberWorx program to help resolve the Air Force’s operational challenges, and fulfilling the CyberWorx vision to unlock the power of people to unleash the power of cyberspace for America’s Air Force. “These types of results are exactly what the Air Force was looking for when it stood up CyberWorx,” said Col Jeffrey Collins, CyberWorx Director. “The cadets have learned to tackle a tough problem by building diverse teams, talking to Airmen and thinking forward quickly.”  

Said Col. Lantz, “I’m glad the cadets are beginning to think of the MDC2 problem set now. These are difficult issues they are helping us solve and they will have an operational advantage as they become new officers in the United States Air Force.” The #AFMDC2 cadet project results will be reviewed and a report available for public release in early 2018.     


AF CyberWorx has released the #radAF tag to show its focus on accelerating human-centered solutions which best resolve Air Force problems. When warfighters have problems fulfilling their duty, the Air Force mission is in trouble. To mitigate this, AF CyberWorx puts the warfighter first as the ultimate user of each solution to Air Force problems. RAD reflects how to best meet and exceed the needs of the user:

Resolve – Each problem can be stated in terms of the basic needs of the user. The user needs must be resolved.

Accelerate – Once the right problem is identified, a rapid solution is needed. AF CyberWorx accelerates the process to a rapid solution concept.

Deliver – Developing the concept is only one piece of the puzzle. Potential challenges and necessary policy and procedure changes are identified to outline a path to solution delivery.

Those three actions – Resolve, Accelerate, Deliver – lead to the Air Force having the agility to identify and respond to each problem in a rapidly changing global environment. That agility is a key piece of what makes the Air Force more effective, ready, and lethal.

*The postings on this blog reflect individual team member opinions and do not necessarily reflect official Air Force positions, strategies, or opinions.