Meet the Team: John Roberts, UX Designer

Meet the Team: John Roberts, UX Designer

Our feature series “Meet the Team,” gives our readers the opportunity to take a deep dive into what makes our team special. We asked John Roberts, our newest UX Designer at Air Force CyberWorx, to introduce himself to our readers and tell the story of his entry into the UX Design world. Please enjoy getting to know our innovative team members and exploring the people who make CyberWorx exceptional.


How does a 6-foot-6-inch, tattoo-covered ex-soldier enter the world of user experience design? The answer is a winding road of life changes.

My name is John Roberts, and I am the newest UX Designer at CyberWorx. Despite having only started in mid-December, it is clear to me that AF CyberWorx is my home. I love the challenge of solving complex problems through design.

John Roberts, UX Designer, CyberWorx

Searching for Career Passion

It’s important to note that throughout my life, I had consistently been searching for my true passion. I joined the Army after two years of college because I felt the direction I was heading had no real purpose in my life.

I decided to sign up to be a Cryptologic Linguist in the Army’s military intelligence program and ended up going to the hardest school of my life. There, I learned Arabic for nearly two years in Monterey, CA.

This school took students from zero knowledge to fluent as quickly as possible. “Drinking from the firehose” didn’t come close to how that school felt. When I passed, I moved on to my career as a Signals Intelligence soldier at Fort Bragg.

Then, my life took a sharp turn when I had major, career-ending foot surgery.

Career Direction Takes a Turn

With running, rucking, and jumping out of planes no longer an option, I medically separated from the Army. Afterward, I went back to school to finish my degree. This time, I pursued cybersecurity, as I felt it would be a good segue from my military service.

However, I realized I was much more into strategic thinking than the nitty-gritty of the cybersecurity profession. That’s when I took a course in UX Design. Getting to the root of users’ problems, testing hypotheses, and crafting effective solutions became my muse. Immediately, I knew I had found my passion.

I graduated with a bachelor’s degree in Information Science and jumped into a master’s program in Interaction Design and Information Architecture (a fancy way to say UX Design).

Here, my love for the field only deepened. I created my own path forward in the UX Design industry while working on my master’s degree and dove into the world of Digital Marketing.

First UX Design Successes

Beginning as an SEO professional, I quickly rose into management. This deepened my expertise across all forms of digital marketing. All the while, I kept true to my goal of UX in the forefront of my mind. To that end, I convinced my agency to let me create a UX department and test out my skills.

It turned out to be a success. One of my most rewarding UX accomplishments to date was converting a client who only wanted to be involved for a maximum of six months into an ongoing client. We ended up doing multiple projects for them. The client was repeatedly impressed by the design process and the solutions we created to suit their needs.

Career Passion Found: Holistic UX Design at CyberWorx

Five minutes into the interview, I knew CyberWorx was the place for me. The environment is one of pure design and innovation. Everything from the sticky notes and whiteboards, to the comfy chairs in the common area, screams collaboration and teamwork.

I had been craving this sort of pure design environment. The CyberWorx team was so intelligent and well-spoken during my interview that I couldn’t help but picture my career here.

With a background in mostly digital UX design, I was excited to help the Air Force and other partners with problems ranging from organizational to digital. AF CyberWorx will help me create a more holistic view of user experience design, while continually challenging my methods of thinking through problems from a human-centered design perspective.

At the end of the day, every designer just wants to solve problems for others. That’s why our industry is so focused on understanding the user, their issues, and their needs with any given system. Our problem-solving style is equally effective with both digital and non-digital problems. I had never considered this before CyberWorx opened my eyes.

In the few weeks since being here, I have already spoken with high-ranking officers, contributed to critical CyberWorx projects, and expanded my understanding of Air Force structure. I am excited about our current projects and our various approaches to get to the heart of the problem and craft an effective solution.

Family Support

I also truly appreciate (and want to brag about) my amazing support system: my wife and family. Savannah has been a driving factor in pushing me to pursue my passion for UX, as well as my non-career passions, including powerlifting, streaming, gaming, movies, comics. Along with my two dogs and two cats, my family keeps me on my toes and ready for anything.

Continued Growth in Crafting Solutions

I originally got into UX Design simply because a college course really spoke to me. My wife loves to say that no one gets more frustrated about an unusable website than me. That spark has turned into a lifelong career and passion for crafting solutions.

CyberWorx is somewhere I can contribute in a meaningful way to the overall mission and continue to increase my knowledge from my peers, projects, and systems. This goofy, geeky dude has found a home with like-minded design professionals, and I am beyond excited to see what the future holds!

Visit our Teams page to learn more about the rest of our team and be sure to keep an eye on our Blogs for our next “Meet the Team” feature in the series!

Futures Research at AF CyberWorx Defines the Problem for Creative Solutions

Futures Research at AF CyberWorx Defines the Problem for Creative Solutions

Futures Research at AF CyberWorx

CyberWorx Futures Graphic

At the Heart of Our Process

CyberWorx helps people solve complex problems. More than that, though, we make sure that when we are solving operational problems, they are the right problems given all the circumstances.

Our human-centered, futures-focused approach reorients problems in a way that puts the people directly affected by the problem at the heart of our process. To do this, we employ several research methodologies, depending on the nature of the presenting problem. We use the knowledge we gain from the user community to solve the problem.

We all use inquiry as a method of navigating the world. We wonder why the grass is green and discover chlorophyll; we wonder how our talent management decisions might affect retention and discover possible trends. Questions are a fundamental component of what it currently means to be human. But not all questions are the same, and not all inquiry is equally effective. Research shapes our inquiry and enables more robust discovery.

CyberWorx Distinctives

The Air Force has a number of research organizations, from AFRL to the Skunks, each with a different perspective on the way they conduct research. When we describe ourselves as a research and problem-solving unit, we are including our organization in this group. Yet, if we were all the same, there would be little use in having multiple organizations. How is CyberWorx different?

We Find the True Problem

Within research methods, AF CyberWorx focuses on generative and exploratory research. What does that mean? When we are helping an Air Force organization, we are helping them explore the problem space to become more knowledgeable about the root cause of their problem. We often use user experience research methodologies; however, for certain problems we also employ research methods from futures studies. 

We Focus on the Future

Futures studies comes from a long tradition of using our understanding of the nature of the future to inform the present. However, the goal of a futures practice is not to predict the future. Instead, the goal is to use future research and methods that give next steps to account for the nonlinearity of the future.

When to Use Futures Research

When would you employ futures methodologies? As the name implies, a futures practice is most valuable when an organization needs to understand how to move forward—or gain a greater appreciation of what forward even means.

When coming to AF CyberWorx with a problem in mind, you should consider two main dimensions: time and outcomes. If you are constrained by time, that will constrain the amount of research you can accomplish. If you need more understanding of a problem that includes past and present, or help with a product or process, you will want to consider other research methods, such as user experience research. For outcomes, if you need help crafting strategy, this is well-suited for futures research.

Futures Word Cloud CyberWorx Graphic

The Five Steps of Futures Research

Five major steps frame our futures practice here at AF CyberWorx: horizon scanning, problem definition, scenario making, analysis, and strategic output. It is important to know that while some of these can be conducted individually, they produce the best results when practiced together. These are not purely linear but can lead into prior steps or circle back to the beginning depending on the nature of what is learned at each step. 

1. Horizon Scanning

Horizon scanning might be the most familiar. It is the process of looking at various sources of information (research papers, news, patents, etc.) and compiling them in a meaningful way. Different futures practices compile these scanning hits differently, but at AF CyberWorx, we often use a thematic approach.

A trend reported on the news can be valuable. However, what is more valuable are the themes and/or values that are fueling the trend. You might read a news article about gradually improving quantum computing technology. While it would be tempting to put that improvement on a graph and use linear predictions to see how that trend looks in the future, this doesn’t reflect reality.

Instead, if you realized that the rise of quantum technology has everything to do with combining our increasing understanding of nature with the technology that best gives us efficiency (computing), you can then see further possibilities of how an understanding of nature might affect computing (or any other technology). 

Historical research brings in data from twice as far back as you are looking forward. For example, if you’re looking 10 years ahead, you should look back 20 years for historical data. After horizon scanning and historical research , you are nearly ready to add some firm definitions around your problem.

2. Problem Definition

To best define your problem, first speak to the people who experience it. We conduct interviews at AF CyberWorx, though we are agnostic to the exact process. For example, some problems may require more observation than interviewing. Compiling the results at this stage will result in a clear understanding of what the problem is, or at least a general domain of the problem. Now you can do something with this understanding. 

3. Scenario Making

Scenarios are one of the core features of a futures practice. It is the art of taking all the data explained prior to this stage and crafting a story of the future from it. It is a sort of alchemy of turning data into creative stories, which will then create insight.

In many practices, a team of writers often connected to the team of researchers will write the scenarios. These scenarios will then be presented to stakeholders of an organization for them to engage with strategic implications.

Where there is merit in doing so, AF CyberWorx employs a workshop method of creating scenarios. We assemble a group of subject-matter experts from government, industry, and academia and leverage the diversity of their perspectives to write the scenarios. Typically, we subdivide the group into smaller groups to generate multiple, disparate scenarios. 

4. Analysis

Rather than present the scenarios as is, we do thematic analysis of the scenarios. Why thematic analysis?

We have just used a workshop method to generate scenarios, and now to harness the power of these crowdsourced stories, we need to explore the similarities. Here’s an example.

Let’s say you broke out the main group into four groups, and each group comes up with a vastly different scenario. We could say one is a doomsday scenario for your organization, another might be a utopian vision, etc. Which has more value to you: four visions of the future that tell you different possibilities, or a set of recommendations based on the overlap of the scenarios? Thematic analysis finds those thematic overlaps. 

5. Strategic Output

The themes then become the basis for various strategic recommendations: plans of action, vision statements, and the like. No matter the format of the output, it specifies what actions the stakeholder(s) need to take next.

It is the difference between a check-engine light and talking to your friend who happens to be a mechanic. The check-engine light will tell you that something’s wrong; your mechanic friend can isolate the problem. In the same vein, our strategic outputs can provide more granularity than simply stating something is wrong with the organization, but they will also outline concrete steps to move toward a future desired state.

Want to learn more about how AF CyberWorx solves problems? Check out our Previous Projects! Keep up with us by following AF CyberWorx on LinkedIn, Facebook, and Twitter.

The United States Air Force (USAF) – Innovation Through the Ages

The United States Air Force (USAF) – Innovation Through the Ages

A Salmson 2A2 of the 1st Aero Squadron over France during WWI, 1918.
A Salmson 2A2 of the 1st Aero Squadron over France during WWI, 1918. (Photo Credit: United States Army Air Service)

On Sept. 18, 1947, the United States Air Force (USAF) was born on the promise to innovate, accelerate and thrive. This promise has led USAF through 75 years of growth and change, bringing about cutting-edge technology and fostering an environment of leaders. 

USAF got its humble beginnings in 1907 as part of the United States Army Signal Corps (USASC). In 1909, USASC purchased its first aircraft from the Wright Brothers. The 1st Aero Squadron, now known as the 1st Reconnaissance Squadron, was formed soon after in 1913. It is the military’s first flying unit, as well as the first unit of the U.S. Army dedicated exclusively to aviation.

World War I & World War II

When the U.S joined World War I in 1917, the 1st Aero Squadron took charge of airfare militarization for the U.S. Army. It was during this time that the U.S. government began to see the importance of tactical airfare. World War II emphasized the gravity of aircraft warfare and the need for an aviation force became increasingly evident.

By the culmination of World War II in 1945, the Army Air Forces had grown spectacularly in size and had more than proved its weight as a major military organization. The National Security Act of 1947 finally solidified USAF as its own military power, separating it into its own branch of government.

Notable Figures

William “Billy” Mitchell was an Army Brigadier General commonly referred to as the “Father of the United States Air Force.” Mitchell served in WWI, where he was appointed Chief of Air Service and Chief Group of Armies.

Left: Army Brigadier General William Billy Mitchell Right: Army Major General George Owen Squier
Left: Army Brigadier General William Billy Mitchell
Right: Army Major General George Owen Squier

After the war, Mitchell was a loyal advocate for air power and believed that the U.S. government should invest more heavily in militarized airfare. Mitchell’s staunch, and occasionally erratic push for strengthening air power, ostracized him from the military community, which led to his resignation from the U.S. Army in 1926.

Posthumously, he was promoted to Major Gen. by President Truman, and his personal sacrifices paved the way for innovation in the Air Force as it is known today. 

Another notable figure in Air Force history was Major General for the U.S. Army, George Owen Squier. Squier served in the Signal Corps until 1916 when he began as the head of the Aviation Section. Even before his time at the Aviation Section, Squier was involved in aviation and worked with the Wright Brothers to prepare the first military plane.

Among Squire’s notable inventions also includes the development of phone networks, which he coined “wired wireless.” He was also the inventor of Muzak, which transmitted music through powerlines and telephone wires, although he died of pneumonia in Washington, D.C. before he could see it become a success.

Innovation & Technology

Innovation has become ingrained into the history of the Air Force and is widely known as one of its many important functions. USAF has an efficient way of handling obstacles and developing technology to suit the users’ needs.

Many widely used technologies originally created to solve an Air Force problem, are now commonly found in everyday life. For example, the Global Positioning System (GPS), as well as drones, can be credited to the Air Force. USAF can also be credited for many of the safety and efficiency functions utilized by commercial airplanes.

AF CyberWorx is proud to be the U.S. Air Force’s Leading Problem-Solving Unit, with innovation and collaboration being the lifeblood of our organization. Learn more about how we solve problems for the U.S. Air Force and feel free to reach out with questions! 

The Case for Purpose-Built Software

The Case for Purpose-Built Software

An Air Force Contracting Officer (CO) in Romania is tasked with developing business relationships and arranging contracts with local vendors throughout the country so that when the need arises to quickly build a temporary runway, such as to provide humanitarian support in the event of a natural disaster, contracts are already in place. The officer meets with vendors to determine the availability of locally sourced products and services such as concrete, catered food, porta-potties, etc., so that the Air Force can avoid flying everything in on transport aircraft. Why spend the money to fly in 4,000 tons of gravel if you can buy it from local vendors?

Though the CO is the person making contract agreements, it takes a team of planners, managers, contracting officers, and contract observers to make this work. Until recently, this contracting process, all done with spreadsheets and documents, was fraught with errors and duplication of effort.

Create Task-Specific Tools

Rather than a typical generalist tool with a single interface, AF CyberWorx developed 4 separate yet integrated tools for the different user types (planner, contracting manager, contracting officer, and observer) and their specific set of tasks (task-oriented design). Planner tasks demand a large workspace, such as a desktop with multiple screens. Contracting managers work from remote locations like hotel rooms and require laptop-sized screen designs. Contracting officers work in the field with tablets. Observers only need a smartphone app. 

The result is an enterprise-oriented, integrated suite of tools that focus on specific users and tasks and eliminate unnecessary data. Not only is each tool optimized for discrete user tasks, but the system also includes algorithms to perform many calculations and actions automatically, relieving much of the users’ cognitive burden. 

Reduce Cognitive Burden

This new approach is a welcome departure from traditional software designs which feature a data input paradigm. These “solutions” are simply information repositories—users perform the real work in their heads and then enter the results into a form. Form-oriented designs only address superficial aspects of a task, treating the system as a knowledge-capture tool rather than a knowledge or solution generator.

Good user research identifies the cognition occurring in the users’ heads and translates those computations into work a system needs to accomplish for users. A proven successful approach identifies best-practice knowledge or processes and builds them into the system. 

Flexibility is the antithesis of best practice or knowledge design.

Raise Users to a Best-Practice Level

For instance, when a group of users develops a more successful (best-practice) process for scheduling their assets and personnel, optimizing the task flow to promote that best-practice approach ensures that every user succeeds to that same best-practice level, often beyond their individual capabilities. In contrast, a flexible design (think spreadsheets) is a form-based solution that relies on the users doing all the work in their heads. In general, flexibility is the antithesis of best practice or knowledge design. Flexible interfaces rely on the users knowing how to use the tool as well as knowing how to perform their tasks well. A flexible tool typically requires that the users adapt their cognition to think from the tool’s perspective—how would the tool work best to do this thing—which increases the users’ cognitive load and further encumbers the task.

Stimulate Solution-Driven Actions

The ubiquitous dashboard is another common example of a design that fails to unburden the user. A typical dashboard is a compilation of static, moment-in-time data, albeit with colorful charts and graphics. What do these data incite users to do next? Users perform better with a purpose-built dashboard that displays trends and exceptions coupled with suggested actions that address the identified issues.

In a simplistic example, a company sales rep needs to meet their sales numbers to earn their maximum bonus. The company has developed best-practice formulas that suggest specific actions to successfully increase sales. A purpose-built dashboard indicates that a rep’s sales are trending below the “bonus” target and presents best-practice options to solve the problem. The suggested actions focus on actions to increase sales numbers.

Purpose-built designs standardize processes and reduce dependencies on individual user knowledge. The sales rep could determine this on their own, but then their success is dependent on highly variable levels of individual skill and knowledge. Any design that relies on such highly variable skills and knowledge cannot succeed at a level any greater than that variability. Such a user-dependent approach suffers from individual inconsistencies and fails to leverage proven, best-practice knowledge. 

Redefine the Problem, Address the Root Cause

A common but ineffective design approach to complex problems is to transform current tools and workarounds into apps—i.e., automating spreadsheets. Successful purpose-built solutions redefine the problem from the users’ perspective to address the root cause rather than solving the symptoms. User Experience research accurately redefines the problem and leads to truly purpose-built solutions. 

Though the purpose-built Air Force solution was complex to design and build, user research led to solving the right problem, saving the Air Force more than just time. This new solution is quickly earning a reputation for simplifying vendor research and eliminating user errors while achieving better results. It is quickly becoming the standard by which other software is measured. If you want to dominate a domain with a cutting-edge solution, don’t automate current solutions; invest in thorough good user research to build a task-oriented, purpose-built solution. It drives better outcomes!

Creating a Standard Scheduling Paradigm Across the Air Force

Imagine a day when a squadron’s flight scheduler arrives at the office and creates the following week’s flight schedule before her coffee even gets cold. Then one of the pilots scheduled for tonight’s inflight refueling exercise calls in sick. She marks the pilot as “unavailable,” and the system recalculates to schedule another pilot who needs night refueling practice. Total time on task: 5 minutes.

It requires immense effort to develop a complex schedule that meets all the training objectives and priorities to prepare a squadron, group, wing, or the Air Force for its primary role. With today’s current tools, scheduling requires one to two people to work all week. Last-minute rescheduling is even more difficult, resulting in scrubbed missions or training. Given that the Air Force is using a myriad of tools and processes to solve this problem with disjointed apps, puck boards, and spreadsheets, complex scheduling tasks are left to be performed in the heads of humans. These are not so much scheduling tools as they are schedule-capture tools; rather than creating a schedule, they only provide a way for the user to capture a schedule already created in her head. In addition, lack of standardization results in a steeper learning curve when aircrews move to new units, making it difficult for schedulers to assist sister units.

How can the Air Force solve this ubiquitous and complex problem?

AI to the Rescue

As part of the AF CyberWorx User Experience (UX) team, we conducted Human-Centered Design (HCD)–focused observational user research at several squadrons and discovered that units typically solve the same scheduling problems with custom solutions, custom apps, custom spreadsheets, and always incongruously. The biggest issue was that regardless of the tools used, the process placed high demands on user cognition, thus relegating each solution to the limitations of each scheduler’s capabilities. A solution to this problem relies on reducing the demands on human cognition by developing a system that can balance the complex needs and priorities to create a workable schedule.

We have identified a single, enterprise-wide Smart Scheduler Paradigm that can be applied to various scheduling domains across the entire Air Force—such as pilot scheduling, Airman training, and readiness forecasting/preparation.

The Smart Scheduler Paradigm (an AI algorithm) puts the emphasis on designing the system to do the work for the human, effectively reducing the cognitive burden of balancing varying assets and personnel needs. After a system is primed with rules-based objectives, assets/resources, personnel, and other adjustable parameters and priorities, the system can automatically generate a suggested schedule. Users will be able to adjust the parameters, such as indicating a pilot is out sick, which can trigger the system to recalculate the schedule using the prioritized objectives. This Smart Scheduler Paradigm will reduce the typical pilot scheduling efforts from about 60 hours per week to about 30 minutes. Other scheduling tasks will likely see similar benefits.

Smart Scheduler Paradigm

The Smart Scheduler task flow is quite simple, only requiring that users define the different elements of the system, objectives, and priorities one time. The system is ready to calculate a new schedule with each parameter adjustment.

Smart Scheduler Task Flow

Poor Technology Maturity

The Air Force’s current approach to technology design and development tends to focus on the technology without much emphasis on the end user. Instead, we adapt the technology to the user. While the rest of the commercial world embraces HCD principles to improve user and organizational successes, the Air Force focuses on technology risk versus the risk of fielding capability that forces the user to adapt to the technology.

Many innovative “solutions” are higher functionality spreadsheets. While many AF members have PhD’s in “Spreadsheet-ery,” users are still left with the cognitive burden of developing schedules while the spreadsheet serves as the knowledge repository. Many spreadsheets are often digital representations of paper or whiteboard tools from the ’60s, ’70s, or ’80s that did little to address the problem or advance the solution. A hallmark of good UX design is getting the system to do more of the work for the user rather than just capturing human outputs.

Commercial companies recognized the cost of non-standard user interaction and interface design models decades ago. They adopt and implement HCD strategies to create cross-platform standardized interaction models that leverage repeatable interaction. These models promote immediate success when a new user tries a new application and reuses familiar interaction models.

Training Addresses the Symptoms, Not the Problems

The Air Force tends to rely on training as a solution for poorly designed technologies. For example, just about every unit completes scheduling tasks with a custom scheduling solution. Every solution is different, requiring new users to waste an inordinate amount of time learning new tools for the same tasks.

It makes sense, then, for the Air Force to commit to developing an enterprise-wide, common scheduling interaction model that could drive the design of every scheduling app. This approach would incorporate greater UX maturity, saving thousands of hours of training time and millions of dollars in development costs.

Institutional Knowledge/Best Practices

A key benefit of a common design paradigm is that it captures best practices from shared institutional knowledge across the Air Force rather than the disparate knowledge specific to individual solutions, driving systemic improvement and efficiency gains rather than incremental improvements at the unit level.

Next Steps … ?

Create a simple mock-up to see how this paradigm can evolve the Air Force into a learning organization and produce a blueprint for other enterprise-wide solution approaches. How many other problems can be solved with such a Human-Centered Design approach?

Project Update: CWS Inform

AF CyberWorx Project Update: CWS Inform

Written by Lorren Stahl, Technical Writer
and Clara Cirks, Marketing Specialist

CWS Inform Project Background

In 2018, AF CyberWorx began a project to assess current readiness management systems for 621st Contingency Response Wing out of Joint Base McGuire-Dix-Lakehurst, New Jersey. AF CyberWorx interviewed Graduate Training Integration Management System (GTIMS) users to understand user needs and assess the ability of the GTIMS. Our User Experience (UX) Designers solicited feedback on user pain points, user journeys, and needs of an ideal system.

Key Factors Discovered in GTIMS User Interviews

AF CyberWorx team members discovered three key factors during this series of interviews:

  1. Readiness is a critical factor for squadron commanders.
  2. Commanders rely on the readiness information they receive to be current and accurate.
  3. Current systems do not collate information, making readiness assessment difficult because information needs to be verified through more reliable sources.

Additionally, data sets pulled from GTIMS were disparate and lacked actionable intelligence. As the stakeholder’s goal was to enable commander/staff readiness orchestration and expand capabilities of the system, further development of solutions was integral. Small business CyberWinter Studios was introduced to AF CyberWorx and project stakeholders through a Small Business Innovation Research (SBIR) program to further develop solutions for the platform. The SBIR program enables small businesses to engage in federal research and development for technological potential.

Current Stages of Developed Solutions

Following the discovery phase, CyberWinter Studios designed and developed a minimum viable product (MVP) that consolidates relevant data into a single, intuitive format. This product and partnership are still active, and currently, CyberWinter Studios has a Phase II SBIR award. CyberWinter Studios’ current product, CWS Inform, has potential for enterprise automation, as the platform and code provide an interactive dashboard, automated workflows, and normalized back-end data. Code from this project has also been pushed to Github for common use by developers looking for a platform to understand and measure force readiness. Thank you to our partner with CyberWinter Studios, Mr. John Grigg, for continuing to push efforts to engage crucial technology with warfighters.

To learn more about AF CyberWorx initial efforts for this project, please visit the CWS Inform project webpage.

For more information on CyberWinter Studios, please visit the CyberWinter Studios website here.

Images courtesy of Mr. John Grigg, CyberWinter Studios

NPS & AU Grassroots Distributed Innovation Sprint: An Interview with TSgt Daniel Hulter

NPS & AU Grassroots Distributed Innovation Sprint: An Interview with TSgt Daniel Hulter

Written by SSgt Austin Wiggins

To accelerate change within the Air Force and Space Force, we must be able to equip potential innovators with education that promotes innovation at the lowest levels. Recently, CyberWorx had the fortune of hosting representatives of; Air University (AU), Naval Postgraduate School (NPS), Army 75th Innovation Command, Defense Innovation Unit, Army Space and Missile Defense Command, U.S. Navy, MIT Lincoln Laboratory, Tecolote Research, and Lokahi LLC. Despite being from disparate groups, they had the common goal of fostering grassroots and distributed innovation through education and connecting academic initiatives to address operational problems. The common understanding in the group; something had to change.

To uncover what made this session interesting and to highlight some hopes for the future of this project, I interviewed the facilitator of the event, CyberWorx member TSgt Daniel Hulter.

What were your expectations for the group going in?

I wasn’t sure what to expect from this group. It was clear they had a lot of topics they wanted to address regarding educational institutions’ role and needs as part of the defense innovation ecosystem. What wasn’t entirely clear was which of these topics were universally felt or what the priorities were. So, I knew that one of the first things we had to accomplish was getting on the same page about what needs were shared and which were more confined to a particular group.

One thing I did expect was that the conversation needed to be driven forward in order to move from observation and analysis to action. We generally have a high appetite for exploratory discussion, especially those in academic circles, and that can make the transition to concrete action challenging. Some participants advocated for less structure to the session so that loose discussion was allowed to happen freely for longer periods while they were together, something we attempted to balance against the desire for clear courses of action. 

In what ways were those expectations positively subverted?

The group performed very well with the more structured portion of the workshop, which I won’t say subverted my expectations, but one of the things I was nervous about was whether they would appreciate being time-boxed and asked to perform very specific analytical tasks.

There were particular exercises that the group took to extremely well, for example their exploration of the defense innovation ecosystem through the lens of analogous systems. I remember feeling relieved that I wasn’t having to goad or micromanage them to do the exercises, which can be the case with certain groups.

What were your thoughts about the group’s objective to enable grassroots innovation through education?

I was slightly caught off guard by the inclusion of the “grass roots innovation” language in their problem statement. I realized that had I been more involved in the crafting of that problem statement, I might have been better prepared to speak to it. Instead I simply put it in front of participants and saw how they chose to interpret and include it in their exploration of the topics that came up.

I do think that the inclusion of that language speaks to the motivation of some of the organizers of this event to try and approach innovation education from a new angle, looking at distributed effects rather than targeted, exclusive, role-based impacts. But one thing we did run into there was that some of the individuals who helped craft that language did not join us for the session itself, which further limited the degree to which that problem statement guided participants in the direction they took.

I think there’s an opportunity in the motivation within this community to do a bit more to try and reimagine what innovation education could look like. It feels like what we accomplished in this day and a half was some broad scoping and a picture of some of the primary systemic mental models that are driving the current shape of the education system. One of the NPS hosts mentioned a paradigm shift around how we even think about innovation education and that’s really interesting.

But this wasn’t a workshop with a precise focus, it was much more exploratory. That question of how we might subvert or mutate or evolve the existing model to create something that is more aligned with the need for ground-level, grass-roots innovation is something that I think deserves significant time and attention and might be an interesting next step.

What are some outcomes you think that the group came to?

This group first achieved a few rough sketches of the current perceptual foundations of innovation education development and delivery for AU and NPS. Using that as an anchor, they thought about ailments within that existing system. They also arrived at a few measures that might surmount impediments to their primary value being delivered to more of the force. The proposed measures they came to at the end of day one can serve as launching points for experiments for these organizations to run, either together or separately.

Much of the time spent together in this session was also spent in discussions. I am confident that these interactions had a positive impact on the participants as they navigated the issues that came up during exercises on day 2, which was almost entirely unstructured. One of the common themes I heard come out was that this exact type of thing–coming together in one place and attempting to make sense of disparate and shared experiences across the defense education community–was something that ought to happen more frequently and perhaps with an even wider variety of participants. I think this may turn out to be one of the more significant outcomes, as it has the potential for continued delivery of insights, increased alignment, and sense-making, and increasing the likelihood of success for experiments that emerge from engagements like this.

What would you like to see from the organizations in this group in the future?

In the future, I would like to see these organizations do a few things:

Come together more frequently and invite more participants in, both with the type of forum that we created with this event and with more continuous connection mechanisms facilitated by platform selection/development, community building, and community management.

Spend more time thinking about and critiquing the perceptual foundations of their current organizational structures and strategies and put some dedicated time into seeing how those structures might be reimagined and redesigned.

Move forward with the experiments identified in the session as potentially high-value by giving them time and space in their own design efforts.

For those organizations who are seeking to empower grassroots innovation, do you have any recommendations?

My number one recommendation is to identify what is within the adjacent-possible for those you are expecting to innovate, for your particular context.

Every organization, unit, and team in the military has its own set of unique conditions and constraints that mean that the approach to enabling innovation and the form that that innovation takes has to be adapted to that level, a task which requires significant time and skill. Rather than seeking to identify standardized adaptations that might work for every circumstance, or even focusing on primary constraints of the larger system, I think it makes more sense to widely teach the very few underlying principles and practices that are universal. To try and affect a culture of acceptance, safety, and experimentation, and spend your remaining energy and resources on enabling that discovery and adaptation to happen in all the places it needs to happen. 

If we are talking about true ground-level innovation, speaking as someone who has spent a majority of my career at the tactical level, I think that means enabling people to be sense-makers and practitioners within their own context. Those practitioners can be plugged into more strategic and operational-level scaling and implementation efforts, so creating systems that transition and scale outcomes should be a priority for organizations as well. In my opinion, empowerment of grass-roots innovation doesn’t start with the question of “are we getting things across the finish line” from a top-level perspective. It starts with creating the opportunity for individuals to contribute and have actual impact on their immediate environment.

Another recommendation that I have is to tell more stories of failure. One thing that came up during our workshop was the fact that shame is a powerful motivator. We are never so innovative as when we are ashamed because we allowed a terrible disaster to occur. The most incredible transformations are possible when we actually reflect on the current state of things and the ways in which we are failing (either by choice or when forced by some kind of incident).

A mistake I see a lot of organizations making right now is telling too few stories about failure. We regularly identify the need to normalize and embrace failure within our culture in order to spur innovation, but we still don’t talk about it enough, especially in the context of innovation (I think largely because we feel the need to say we’re succeeding in order to keep getting funding and support from our organizations and leaders). Lots of efforts at enabling innovation are failing, and we need to be open and honest about that in order to create space and energy for the next pivot.

For the record I am guilty of this exact thing. Storytelling and marketing are crucial components of success in all ventures, especially innovation. Building coalitions requires that we convince others of our likelihood of future success, and claiming present or past success is a potential pathway to that, but it suppresses stories of failure that might allow us to be ashamed enough of the status quo enough to drive the change we need.

Do you have any closing thoughts?

It was wonderful to see disparate players from across the innovation ecosystem come together and navigate these difficult topics together, in both structured and unstructured ways, and I hope to see this exact type of thing happen more often.

Thank you Daniel Hulter, for taking your insights into the session. With your participation, we are one step closer to delivering real qualitative change to grassroots and distributed innovators throughout the Air Force.

For more information on this project, visit the NPS & AU Grassroots Distributed Innovation project webpage.

CyberWorx 2021 Quarter 1 Newsletter

CyberWorx Newsletter 2021 Quarter 1

Team Comments

The chemical company, BASF, used to have an advertisement that said in part, “…we don’t make the cooler, we make it cooler.  We don’t make the jeans, we make them bluer,” to highlight how their company improved the experience, performance, or durability of products for the people who use them.  Much like the ubiquity of the products presented by the advertisement, information technology now underpins nearly every aspect of daily life both at work and at play.  In a corollary, AF CyberWorx exercises their human-centered/user experience design, lean startup, and agile toolsets on the application of technology to improve experience, performance, and efficiency of mission execution, preparation, and support, while helping prepare the Air Force for new/emerging technologies and aiding industry in better understanding operational needs.  

Our Cyber Risk Ecosystem, presented in previous newsletters and currently in integration/deployment with NORAD/NORTHCOM, provides cross-domain risk awareness to help commanders understand and mitigate cyber threats to missions in other domains.  The recent low/no-code initiative for 16th AF demonstrated the efficacy of equipping unit-level Airmen with tools to automate and improve their own processes such as aircrew training and mobility readiness and awareness, network account request automation, contracting officer/small business interactive marketplace, personnel functions, and training instructor scheduling.  Our team has helped the F-35 JPMO improve mission communications planning, and we see the potential for that work to expand across multiple platforms and across the Joint Services. 

At AF CyberWorx, innovation isn’t just about technology, but a balance of meeting user needs within organization constraints with the right technology to make Airmen more effective at the full spectrum of activities needed to deliver Joint capabilities from mission support through mission preparation and execution.  At AF CyberWorx, we don’t make the Commander’s decisions, we help make them better informed and more quickly.  We don’t load the cargo, we make tracking it more efficient and accurate.  We don’t fly the mission, we make the planning easier, faster, and more precise.

Lt Col Helgeson, Deputy Director

CyberWorx 2021 Quarter 1 Project Portfolio


The Air Force Test Center wants to accelerate the data processing pipeline to move useful, actionable test range data to decision makers faster. CyberWorx is working with Hill AFB’s EDDGE software team to design and develop MVP screens for an application that streamlines access to a centralized data platform.

ai cyber wingman

The Cyber Wingman project is focused on enabling an AI-based mission commander digital assistant that can compile and process large amounts of data into actionable information, including similar past events and recommended courses of action. The CyberWorx team is working with MIT and Lincoln Labs, planning and coordinating research and interviews. 

f-35 enhanced ui

CyberWorx is providing expertise to a joint work force of Navy, Air Force, and Marines with a deep dive into task analysis as they update their mission planning software that aligns to a next-generation framework with a modern user interface.

USSF Enlisted talent development

CyberWorx is working with USSF to identify how to best develop fully qualified enlisted Guardians that will be ready to meet future Space Force challenges.

leading user experience for everyone (Luxe)

The Chief Experience Officer of the Air Force is spearheading an effort to improve the form and function of existing enterprise-wide applications. The LUXE project underscores the importance of UX when developing products and services and highlights the need for wider adoption of UX practices across the Air Force. The CyberWorx team is providing design consultation and resources for education and training on human-centered design methodology as well as the redesign of some of the Air Force’s most egregiously designed applications.


AF Spark Tank finalist, Kinderspot, will offer families enrolled in Child Development Centers the ability to sublease their child’s spot to other eligible families while temporarily out of the area, Airbnb-style. The CyberWorx team worked with industry partner Oddball to optimize the user experience through the research and prototyping phase.

usafa crsp

 CyberWorx facilitated a design sprint 1-2 March to investigate how to develop a Consolidated Resourcing Sight Picture for the USAFA Financial Management office that would enable effective use of resources and provide better decision support to the Commander.

mission assurance with spectrum

We recently hosted nearly 150 government and industry reps to answer the question of how we might use the variety of Electro Magnetic Spectrum options to provide mission assurance of DoD operations. CyberWorx partnered with the Air Force Spectrum Management Office, the 350th Spectrum Warfare Group and NineTwelve, a public-private partnership in Indiana, to run a 3-day virtual event to connect emerging Spectrum technology with DoD stakeholders/users.

weather ai explainability

The Weather AI project will include explainable AI in future weather forecasting models with an emphasis on overseas locations with limited weather radar coverage to develop high-fidelity and explainable products. We are teamed with MIT and Lincoln Labs researchers on approach and questions for user testing in the effort to better inform the development of next-generation forecasting tools.

the other airmen

The Other Airmen initiative (pilot #1) successfully concluded on 11 March as six Airmen “Citizen Developer” teams pitched their solutions to 16th Air Force Commander, Lieutenant General Timothy Haugh at the Rocky Mountain Cyber Symposium. We are continuing work with the Air Force CIO office and 16th AF to move forward with finding ways to provide the capability to Airmen across the enterprise. 

ussf cyber officer force development

We are assisting the Space Force in developing a more agile, digitally-focused career development path for their cyber officers. 

21st century drill

Twenty-two participants from across government and industry attended a virtual design sprint from 26 January to 19 February to refine problem areas from information gathered through Guardsmen surveys about the collaboration and communication challenges they face getting ready for their drill weekends and annual training. A working group will continue work on the feasible solutions developed during the event to build the Air National Guard drill weekend of the future.

airmen leadership qualities

CyberWorx is collaborating with HAF/A1H as they work to make improvements to the Air Force’s officer and enlisted Evaluation Systems. Through facilitating AF-wide focus group sessions, the data gathered will inform decisions for evaluation system transformations in the future.

usafa superintendent’s honor system review

CyberWorx facilitated a design sprint for the Superintendent-directed Honor System review. Six teams of participants identified 31 ideas to improve the System and encourage cadets to embrace the Honor System as an ideal to aspire to, building leaders of character for the future.

winter is coming

The 319th Reconnaissance Wing and the University of North Dakota have teamed up to build a culture of continuous innovation in the Airmen of Grand Forks AFB, ND. We facilitated a virtual education and design event for them, focusing on improving the living conditions for Airmen living in base dorms by creating an environment that encourages engagement, curiosity, creativity, and inclusion.

norad-usnorthcom (n-nc) innovation and culture

 Government and industry participants explored how to increase the digital literacy of N-NC and create a culture that embraces an innovative mindset.

The Other Airmen Spotlight

The Other Airmen Spotlight

The Other Airmen experiment aims to demonstrate citizen developers from the Air Force and Army can create useful applications using low code/no code capability. The teams are transforming their use cases into working applications to present to the 16th Air Force Commander Lt Gen Timothy Haugh in March 2021.

24 Feb – The process to pull and compare information for a unit member – military or civilian – is labor-intensive.  It requires data to be cross-referenced from multiple sources including spreadsheets and specialized systems. Currently, personnel acting as Unit Deployment Managers, Unit Training Managers, Commander Support Staff, and other positions need to collect this information, compare it for accuracy and currency, and route it up to appropriate leadership.

Two Citizen Developers are collaborating on a unit-level application that imports and aggregates complex spreadsheets such as the Unit Manning Document, Unit Manpower Personnel Record, Alpha Roster, Gains and Loss rosters, and Civilian Roster pulled from personnel systems. They can also manually modify personnel records using “detail screens” which displays specific information on an individual.

The application suggests changes and allows users to manually verify where changes or inaccuracies exist. User access to system data is role-based, granted as their account is setup by an administrator. Role-based access maintains data confidentiality while still providing appropriate access for specialized personnel to gather information quickly for mission-essential tasks.

The application is currently functional with access to real data. The Citizen Developers are adding additional features from their “want to have” list. They are also gathering more information to provide a more robust example for the March presentation. They would like to import data directly from system databases, but current bureaucratic controls require manual spreadsheet imports. This application is a perfect example of individuals identifying a need for a tool that reduces time- and labor-intensive tasks to improve efficiency and accuracy. The sleek design also highlights that even amateur developers can develop relatively sophisticated applications that perform well and look professional in a relatively short span of time.

A screenshot of the role-based token creation for the manning application.

31 Dec – Today’s Spotlight focuses on a Market Research and Solicitation tool designed for contracting officers to efficiently perform market research on a company submitting for a Federal Commercial Solutions Opening. Companies could upload pitch decks including a descriptive video and whitepapers to support the submission. The tool could foster continued engagement with an organization after an event or project is done.

Currently, data in the MVP is entered manually. It includes links to the announcement and any social media platforms the company uses. The citizen developer is seeking APIs and permissions to automatically pull company information using the DUNS identification number, which is a unique nine-digit number required for any company to register with the Federal government for contracts or grants. Other potential sources for automatic information include for contract identification, the Air Force Installation Contracting Center business intelligence unit for Federal Supply Codes and Product Service Codes, and ready-made prospective sheets for the company. The end goal is for the tool to minimize manual entry and provide users current information.

The citizen developer is an Air Force contracting officer currently participating as an Education and Industry fellow as her full-time position. She built the MVP on her own time on weekends after completing her training. The short time-frame from idea to MVP demonstrates how quickly a citizen developer can develop a solution that improves their productivity.

10 Dec – Of the citizen developers volunteering their time and efforts, one has accelerated the development timeline and delivered a prototype. War Skills and Military Studies instructors need a more efficient and reliable way to schedule a complex class coverage. The current process makes a team of instructors unavailable for teaching while they manually build the schedule. That draft schedule is visually compared to each instructor’s leave schedule and the class schedule. If a mistake or unforeseen change occurs, the team has to rush to make updates and disseminate the new schedule to the instructors as quickly as possible. 

The use case involves a scheduling application that allows a single user to easily add classes, instructors, and locations as well as de-conflict instructors with scheduled leave. With the push of a button, the system will process the information and assign instructors to classes. Instructors won’t be scheduled if they’re on leave, nor scheduled to teach two classes at the same time. The system accounts for travel time between class locations. The application quickly produces an equitable schedule and allows the Scheduling Office the ability to easily and quickly make changes and disseminate an updated schedule within minutes instead of days.

That is the power of Low Code/No Code that The Other Airmen experiment is assessing for wider adoption. Citizen Developers know their pain points. Given the tools to develop their own solutions, DoD personnel can quickly improve unit efficiency.

11 Nov – Personnel responsible for member training requirements need a better way to track training data to reduce overhead and create a comprehensive picture of a unit’s ability to support the mission. Currently, commander support staff, unit training managers, and unit deployment managers have to sift through information from an array of sources including ADLS, TBA, IMDS, the Army’s DTMS, and various spreadsheets to piece together an accurate report. Users deserve a more efficient way to track training than cross-referencing 10 different systems to extract required data and manually input the information into a spreadsheet.

Several citizen developers are working on training tracker prototypes to streamline the current tedious process.  An automated process will reduce the amount of time spent manually entering data, improve accuracy, and allow Airmen to focus their time and effort on other unit priorities.

The Other Airmen: Low/No Code Milestone Two

The Other Airmen: Low/No Code Milestone Two

Citizen Developers participating in The Other Airmen low/no code experiment made excellent progress towards functioning minimum viable products (MVP) in December. While the second in-progress review (IPR) in December had three demos, the third IPR doubled that with six demos. 

The big rocks for the developers in December were clearing up lingering AFNet connection issues, expanding what capabilities they had available to them, and discovering APIs for data connectivity between their new applications and existing databases. The most successful capability demonstrated during the 12 January IPR was the seamless integration of DocuSign into an application for document package routing and signing. Other capabilities the developers are incorporating in their solutions include automatic account creation through CAC card registration and ranking data by confidence level. The breadth of use cases under development demonstrates the promise these platforms may hold for Airmen across the enterprise.

Several Citizen Developers are working on their solutions from home because of connection issues through AFNet. The ability to work on a solution from any location is important. We need to equip Airmen to work at their home station, while TDY, or even on deployment as long as they have an internet connection. We anticipate the barriers of going through AFNet will subside if low/no code is adopted as an enterprise approach for solution development.

As the teams pass the half-way mark of the experiment’s scheduled timespan, they transition from planning into development mode. The more mature solutions are now moving from “get it working” to “how can it be better” with more automation, a stronger user interface, or a broader range of database connectivity. The low/no code platforms and the Citizen Developers are exceeding expectations as they code future capabilities for their units.