We know that you’re thinking, “Really? Another fluff post about the top people in my profession and what I need to do to be just like them?”

Listen, we hear you, but this is not some fluff piece with vague ideas of what might (or might not) work. Read on, follow along for the next few weeks, and we promise it will be worth your time! If not, write to us directly and tell us what we can do better next time. We will reply!

In case you missed it, on May 27th the 2019 State of Automated Visual Testing was released. Based on independent research sourced from over 350 testing teams around the world, we learned that 12% of you are getting much better results than the other 88%. We’re talking four times more successful as measured by the things you (and your boss, and bosses’ boss, and your bosses’ bosses’ boss) really care about – test coverage, release velocity, application quality, overall R&D teamwork, and cold hard cash! This is not our opinion. It’s not subjective. It’s objective data and information. Data and information that came from you and your peers.

In other words, we’re not asking you to take our word for it; we’re asking you to take your word for it. Take a moment to download the full report. We will be here when you get back!

Click here to download the report.

So what separates the top 12% from the other 88%?

Digital Transformation. Two words that have been baked into our world over the past 20 years. To such an extent that it may be fair to call it fluff? Fluff or not, IDC forecasts that worldwide spending on technologies and services that enable digital transformation will reach $1.97 trillion in 2022, per the (IDC) Worldwide Semiannual Digital Transformation Spending Guide.

(If you think a trillion is an abstract concept, check this out.)

Dang. That’s a lot of fluff! When your bosses’ bosses’ boss is spending that kind of cash, it’s always worth paying attention. It could be good for your career. As it turns out, 12% of the world’s testing teams did pay attention, and it’s their approach to managing the challenges of digital transformation that have set them apart.

The Testing World’s Digital Divide – Digital Transformation Quantified.

You can read up on The Enterpriser Projects CIO level take on Digital Transformation (warning: fluff alert) here, or you can quantify it for yourself with some simple math and see how you compare to other R&D and testing teams around the world. Got that calculator ready? Here we go…

• How many applications do you have in production today? (Don’t forget those native mobile apps, they really add up).
• How many pages or screens do you have in production on average for each of these applications? (Single Page Applications can be tricky we know, but give it your best guess).
• How many viewport breaks do you support? (The market average is six if you are not sure.)
• How many human languages do you support? (We’re talking about localization here – English, German, Spanish, Chinese, Japanese – not coding languages in case you’re wondering).

Now multiply those four numbers together. Congrats! You have just quantified the digital footprint of your business. People can write about digital transformation all they want, but you’ve just transformed all that fluff into something quite real. It’s probably a big number. Over 90,000 for a typical company and over 624,000 for the largest 30% of companies in the world. Here’s the kicker. You and your R&D team are responsible for managing the visual and functional quality of every single one of those pages and screens.

We categorized the number of screens in production by industry, based on your responses. Here’s what we discovered.

When functional test automation first emerged way back in 1989 with the launch of Mercury Interactive XRunner, it was a much different time. Browsers didn’t even exist. Since then, the browser wars have come and gone and standards are now in place. Applications have grown far more complex with native mobile, dynamic content, single page applications (SPA) and responsive design now an everyday reality. Digital transformation for any size company is officially past tense. We’re not transforming, we’re transformed. And now you have to deal with it, but how?

What Defines a Top 12% Global Testing Team?

Top 12% teams have overcome the massive challenge posed by digital transformation. Like any technical challenge we have ever faced, it started with someone on the team who felt the pain and set out to solve for it. And when they did, good things happened.

Test coverage increased by 60%.

Release velocity became 2.8x faster (even though coverage increased 60%)

Visual and functional quality improved 3x with far fewer escape bugs (even though they release 2.8x times faster)

R&D teams are 4x more satisfied with their visual and functional quality outcome (yea, I’d be more satisfied too with those types of results!)

All of this despite the fact that this 12% of testing teams were managing applications 2.2x larger than the other 88%. In all likelihood, they felt the pain before most of us, and have now led the way for the rest of us. We just need to follow.

Goodbye, But Only For A Week or So.

Today successful continuous management of application visual quality creates competitive advantage. Business leaders know this and are paying attention. As a result, testers are in a better position than ever to be heroes respected by their R&D teams for driving huge value for the companies they work for.

Over the next 10-12 weeks, Patrick and I will release a series of blogs that explain how these 12% of testing teams reinvented their testing approach to deal successfully with the challenges of Digital Transformation. We will get into the details as promised.

Or, if you simply can’t wait and want to ignore my spoiler alert, you can listen on-demand to the webinar Patrick and I hosted together entitled Wrong Tool, Wrong Time: Re-Thinking Test Automation, read the blog and view the slides here, or just reach out to us for help with your testing approach.

Until next time.

Yours Truly,

James the #NotSoEvilMarketingGuy

Patrick the #GuyWhoActuallyHasDoneTestingFor20Years

Find Out More

Check out Applitools blogs about digital transformation.

Read about how Microsoft incorporates visual testing to their DevOps delivery.

Request a demo of Applitools Eyes.

Sign up for a free Applitools account.

The original version of this blog post previously ran on devops.com

The post What The World’s Top 12% Testing Teams Are Doing (That You Can Do, Too) appeared first on Automated Visual Testing | Applitools.

What is Tricentis Accelerate? Tricentis Accelerate (@Tricentis, #triacc), is an annual event hosted by our friends at Tricentis that aims to bring the DevOps, software testing, and QA automation community together for two full days of learning and networking. Speaker session topics cover themes around open source, leadership best practices, testing metrics and more. This year’s conference will be held at Hilton San Francisco Union Square on May 22-23.

This year’s theme is Automation @ the Speed of Change — representing the industry’s most modern testing practices and aiming to educate conference goers on how to transform software testing to accelerate their digital transformation.

We are proud to have our co-founder and COO, Moshe Milman, take the stage at this prestigious event. On Wednesday, May 22 @ 1:30 p.m. PT he shares “Tackling Test Automation Challenges in the Era of CI/CD.” The session takes a deep dive into the top test automation challenges for CI/CD and he shares how successful companies are addressing them. The talk will take place at Breakout – Imperial B room.

We are proud to be Gold Sponsors of Tricentis Accelerate San Francisco 2019! Find us at the expo hall, where we will share the latest end-to-end visual testing and monitoring techniques that support automation and improved visual UI testing through the Application Visual Management (AVM) approach.

One of the latest additions to the product toolkit is Applitools Ultrafast Grid, which helps QA teams create the fastest, easiest path to test automation success. By adding the Ultrafast Grid, Applitools Eyes now includes every component the enterprise needs to quickly succeed in its test automation and CI-CD efforts. Our core test management, auto maintenance and dashboard functionality now allow anyone on the QA team to author, fix, analyze results and maintain tests easily.

And, if you want to learn more from other QA and software testing experts after Tricentis Accelerate 2019, connect with the world-class instructors over at Test Automation University for a free, community-driven collection of educational training courses.

We hope to see you there!

If you don’t get a chance to connect with us at Tricentis Accelerate 2019, reach out or sign up for a free Applitools account.

Want to become a test engineering rockstar? What about money and prizes? Sign up for our Applitools Visual AI Rockstar Hackathon. Click here to learn more.

The post Tricentis Accelerate San Francisco 2019 – See You There! appeared first on Automated Visual Testing | Applitools.

This list of pitfalls to avoid when testing in production serve as my notes on Amber Race’s Webinar: The Joy of Testing in Production

If you don’t already know her, Amber Race is a rockstar. She’s a senior software development engineer in testing (SDET) at Big Fish Games. Through her amazing experiences, she continues to gain wisdom on a subject often thought of as taboo in the world of software development: testing in production.

Now, some people think that testing in production is bad, which results in this common meme:

Amber presented recently in a webinar why testing in production is actually a good thing. These are seven key takeaways from her webinar.

But first, some background: BigFish uses API-delivered services to power its games. Amber knows that real-world use can create test conditions she could never recreate in her lab. This is why it’s essential to test in production — as well as in development, of course. Also, monitoring in production has helped her team pinpoint areas of inefficiency that ultimately led to better results.

Pitfall 1 – Willfully Disputing the Need for Testing in Production

The first pitfall to avoid when testing in production is ignoring the need. Amber noted that there is still a hopefulness that testing ends once code is in production. “There’s a traditional view that once you put your stuff out in the world, you’re done. But, of course, you’re not done because this is the time when everybody starts to use it. It’s really just the beginning of your work when you’re releasing to production.”

There will always be one that gets away…

Every time a customer opens an app or a webpage, they are testing to see if it opens properly, is responsive, etc. Amber notes that even if you *think* you are not testing in production, you already are: “Instead of thinking that testing and production is something that you should be doing, you should be thinking that it’s something that’s already happening every moment your application is being used.”

Every application depends on behaviors and environments that may fail when exercised. And, yet, the complexity of applications demands that they be tested “sufficiently” for behavior with a realistic understanding that testing will not cover every real-world use case.

The pitfall here is being dogmatic about correlating production outcomes with how well your QA team does their job. All unit and functional tests must be completed and show that the application works according to a set of environmental metrics. The team needs confidence that the design works as expected in hypothetical customer environments. That’s not the same as knowing the real world.

Pitfall 2 – Overscoping Tests Needed In Your Sandbox

Sandbox testing can give you a false sense of security. They can also consume lots of resources that provide little incremental value. Amber called out this as the next pitfall to avoid when testing in production.

Testing within a “sandbox” doesn’t fully address the complexity of the number of environments, devices, browsers, operating systems, languages, etc. that exist. Amber exemplifies this point: “I counted up on one of our devices and we had in a certain snapshot of time 50 different Android devices. Forty-five different iOS devices, the combination of the device itself and the iOS version. So there are almost 100 different kinds of devices alone. And, that’s not even considering all the network conditions – whether people are playing on a fast Wi-Fi network, or they’re on 3G or they’re on 4G, or they have some spotty connection that’s in and out. You cannot possibly cover all the client combinations in a sandbox.”

If you are thinking the cloud might help you better replicate a production testing environment, think again, Amber says: “Even if you’re able to replicate your production environment in the cloud, does your test user base match production? For example, in the case of the service we provide, there’s a constant load happening of a quarter of a million players every day playing our games constantly happening in the background. We can’t replicate that in a test environment.”

So the pitfall to avoid is thinking that the breadth of your sandbox testing covers all your functional and behavioral environments. Instead, ensure your coverage at the sandbox level exercises your code under load and one or more hypothetical production environments. To be clear, those must pass. But, your tests will never represent the array of environmental and load factors that will impact the behavior of the production application. If you are still expecting you will catch all issues in QA, you need to let go of that idea.

Pitfall 3 – Not building production monitoring

Amber pointed out that another pitfall to avoid in production testing involves proper tooling. Application developers need the right tools for production monitoring.

She talked about one API that her company was using on one of its largest production games, and this API was using a non-standard interface. It wasn’t JSON or SOAP – it was BSON. There weren’t any standard monitoring tools for this API service behavior – and was jumping all over the place. There were failures in production that could not be explained by logs or other metrics. So, she needed a way to help the team understand what was happening with the service.

She explained that code in test is like practicing your steps preparing for a night out dancing. You work on your moves, you look good, you check yourself out in a mirror. But, once you get the club, the environment turns into “a giant mosh pit of users doing all kinds of combinations and all kinds of configurations you would never ever think of.” What is the solution? Monitoring in production! So, you really need to build monitoring into production.

Here are some top practices Amber suggests:

• Monitor your areas of interest. Start small: “ “You can just start with one thing. Like, add counters onto your API usage and calculate the API usage patterns you observe.”
• Take it to the next level. Amber would calculate the usage by API and figure out API dependencies by determining the order in which they were called. “Just by observing what’s really happening you can prioritize your testing. You increase your knowledge of the software.”
• Know which dependencies are in your control. “Once you have dependency information you can add more monitoring around what’s going on. For example, if you have an authentication solution and that is calling a third-party service you might start by looking at how long authentication takes.”
• Be aware that monitoring has costs. All monitoring will consume resources that require processing, memory, and network.
• And, to get to some metrics, monitoring may need to be built into the code at the development or the operational level.

Pitfall 4 – Not Knowing the Monitoring Tools You Can Install

Yet another pitfall to avoid is a lack of information for tools that can store production metrics.  The easiest type of metric storage is a log file. Amber discussed that one of the most well-known log file search engines is Splunk. Another is the ELK (ElasticSearch, Logstash & Kibana) stack. The value of any of these tools depend on depend on the metrics you need being found in the log files.

The next type of metric storage is in a database. Graphite is a tool for stats extraction and analysis from a database. StatsD and InfluxDB are other tools for database data extraction and analysis. Know that this kind of testing will require additional coding.

Finally, Amber discussed both Grafana and Kibana as visualization tools. These are all tools that can be either purchased commercially or, in some cases, downloaded as freeware.

At BigFish, Amber and her team used Graphite and Grafana for API behavior visualization. An example of this work is below:

Bigfish chose Graphite and Grafana for the following reasons:

1. “The stack was already in use by Ops. “Always ask the Ops team what they’re using,” says Amber.
2. Instrumenting the code was easier than fixing the log writers
3. There is a lot of flexibility in the Graphite query language
4. Templates make Grafana really useful.
5. “Pretty graphs are nice to look at.”

Again, the key is to look at the ability to add tools that are easiest, instrument the right metrics, and begin to monitor.

Obviously, you need to know what to monitor and how best to show the values that matter. For example, If you can install a timer for accessing a critical API, make sure you can graph the response time to show worst cases, statistically. Graphite can show you the statistics for that metric. The 95th percentile value will show you how slowest 5% of your users experience the behavior, and the p999 latency will show you how 0.1% of your users are seeing things.

Realize that 0.1% is 1 out of every thousand. If you only have 1,000 users, you might think you can live with one user having this experience. If you have a million users, can you live with 1,000 having this experience? And, if you have 1,000,000 users and they, on average, call this API 10 times during a typical session, they have more like a 1 in 100 chance of hitting this worst-case behavior. Can your business function if 10,000 customers get this worst-case behavior?

The key is to know what you are measuring and tie it back to customers and customer behavior.

Pitfall 5 – Not Learning From Monitoring

When engineers insert production monitoring, said Amber, they may think they are done. This is the next pitfall to avoid when testing in production: failure to learn from monitoring metrics.

The keys to understanding monitoring are knowing that you need to see what is important. For instance, when you want to observe performance, you need to monitor the latency of calls and their frequency. When you want to drive efficiency, you can look for frequently-invoked calls (such as error handling routines) which imply that your code requires underlying improvements.

After implementing their own monitoring tools and processes, Big Fish Games was able to find concrete ways to improve their system. One of the big takeaways was learning that they had API behaviors that were generating errors and then discovered that the errors did not result in user problems.

Amber noted: ”We would have calls that the game was making that failed every time. The calls failed because maybe it was an older client or we had changed the API. Regardless, it was an invalid call. So that’s a waste of time for everybody, it’s a waste of time for our service. It’s a waste of time for the client. Obviously, this call was not necessary because nobody had complained about it before. So having them remove those calls it’s saving traffic on our site. It’s saving network traffic for the player. It’s savings for everybody to get rid of these things or to fix them.”

Additionally, Amber and her team found that certain API calls caused obvious load issues on the database.

“We had another instance where the game was calling the database every 30 seconds. And it turned out 95 percent of the time the user had no items because the items in question were only given out by customer service. So we’re making all of these database requests and most of the time there’s no data at all. We were able to put some caching in place, where we knew they didn’t have data we didn’t hit the database anymore and customer service would then clear that cache. The point is that this change alone caused a 30 percent drop in load on the database.”

Pitfall 6 – Becoming Monitoring Heavy And Forgetting Value

The sixth pitfall to avoid in production testing involves failing to build high value, actionable monitoring solutions.

Once you have metrics, it’s tempting to begin making dashboards that show those metrics in action. However, don’t just dashboard for the sake of dashboarding, Amber advised. Instead, continue to make sure that the dashboards are meaningful to the operation of your application.

Make sure you are measuring the right things: “Always re-evaluate your metrics to make sure you are collecting the data that you want to see and your metrics are providing value.”

Amber explained a critical difference between monitoring and observability. She provided her explanation: “The difference between monitoring and observability is that with monitoring you might just be getting load average but you don’t know exactly what went into that average, whereas when something is completely observable you can identify and inspect specific calls that are happening within that moment of high-performance issues.”

The key, she concluded, is to value metrics and dashboards that let you act and not just wonder.

Pitfall 7 – Not Being Proactive about Performance, Feature, and Chaos Testing

Once you have instrumented your production code, you can use production a test bed for new development. Amber noted that many companies are wary about running production as a testbed. That fear can lead to the last pitfall to avoid when testing in production – not using your production testbed proactively.

No one is suggesting to use production as a place to test a new user interface or graphics package. Still, especially for new services, production can become a great place to try out new code while real-world users are applying load and changing data.

Being able to do performance testing in production is key, said Amber “The number one best thing about testing in production is doing performance testing in production so you can test before your clients are updated to use a new API. You can put it out on your production service and run a load test on that particular API while you have all the background noise going and you don’t have to guess.”

She shared her thoughts on the benefits of feature testing in production: “[With feature testing] you have the real hardware, the real traffic, and the real user experience is happening just in a box. If it’s something you can turn on and off easily then that makes for a very useful test.”

Even chaos testing belongs in production, Amber explained. “Chaos testing is good when you want to see what happens if a particular host goes down or a particular database is offline. With these sorts of network outage conditions, it’s important to see if your flows are working the way they should. You want to test these in a place where you have control rather than letting the real chaos take over. Who wants to be woken after midnight to figure out what’s going on?”

Final Food For Thought

Given that you’re paying attention and not falling into the pitfalls, get ready to add production monitoring and testing to your arsenal of application tests.

Here are a few more musings from Amber that will get you pumped to start testing in production:

• “By observing you can look for new things to look for and you can do this even without a so-called observer to observability solution. Taking in this information is really important for baking more quality into your services and understanding your services and applications better.”

• “The point is not to break things, it is to find out where things are broken. That’s why we’re doing the monitoring in production. That’s why we’re doing this testing is we want to see what’s happening with our actual users and where their issues actually are and hopefully alleviate them.”

• “Most importantly, explore without fear.”

To capture all of Amber’s inspiration on monitoring and testing in production, watch a webinar of her full presentation, “The Joy of Testing in Production”:

https://youtube.com/watch?v=8-ymeVdNxSE%3Frel%3D0%26showinfo%3D0

To visually test your apps before they go into production, check out our tutorials. Or, let us know if you’d like a demo of Applitools.

Other Posts that might interest you include:

1. Test Your Service APIs – A review of Amber’s course on Test Automation University
2. Automating your API tests with REST Assured – TAU Course by Bret Dijkstra
3. Running Tests using REST API – Helpdesk article by Yarden Naveh
4. Challenges of Testing Responsive Design – and Possible Solutions – by Justin Rohrman

The post 7 Production Testing Pitfalls appeared first on Automated Visual Testing | Applitools.

A few weeks back, Applitools made the trip just up the road from our headquarters to take part in DevOps World | Jenkins World San Francisco. The event is an international event that provides opportunities to learn, explore, network and help shape the future of DevOps and CI with Jenkins. It’s always an exciting and energizing event and this year proved no different as we were able to network and engage with leading minds in the world of software development.

While we were there, James Lamberti, our Chief Marketing Officer, had the honor of sitting down with Jayne Groll, CEO of the DevOps Institute, to talk shop and explore some of the ways Applitools has changed how organizations are able to use visual testing to deliver better UI/UX for their customers. Here are some of the highlights of that conversation. 

Understanding Applitools and How It Fits into Digital Transformations

Jayne’s first question led James to discuss the basics of what the Applitools Visual AI tool is – and how it aligned perfectly with the theme of the event.

“Applitools exists to make sure the user interface is visually perfect on every device, operating system, and screen, whether mobile and desktop,” said Lamberti. “The theme [of DevOps World] this year is transform. And essentially, the issue or the problem that we solve for pretty much everyone in this room is digital transformation.”

While “digital transformation” at times is viewed as an industry buzzword, it is inarguable that businesses of all sizes and verticals are using digital as a means of fundamentally transforming their businesses, in part by building applications to streamline customer interactions. So when it comes to the user interface a customer-facing app, there is truly no brand spared from the impact of what a broken UI can do.

Bad UI Can Keep Your Business Grounded

A real-world example James highlighted how a broken UI can fundamentally affect your business, and its bottom line, was his recent experience trying to book a flight via his mobile device. When it was time to finalize the transaction, the checkout button was unclickable – ending the purchase journey right then and there. As James pointed out, this sort of break in the user experience will cause your customer to switch on a dime to another company – and this is exactly the type of issue that can be prevented by partnering with Applitools.

Exciting Developments on the Horizon 

Finally, Jayne and James touched on some of the exciting news coming out of Applitools – namely our newest product, the world’s first UI version control system.

“Essentially, we have integrated with GitHub and, for the first time ever, you can actually tie your code commits into GitHub directly to changs to the UI. This means that, for the first time, you can correlate code changes to UI changes, and tracking the history of how your UI evolves over time,” said James. “And so now, if there’s a UI issue, you can tie it back to what code commit caused that issue.”

For instance, if you see that an application stopped meeting its business goals a month ago — say, transactions declined, or user engagement slipped — you can look at the visual changes that were made to your app’s UI around that time, in order to discern if the application might be to blame. Furthermore, you know what changes to undo in your code to get your UI back to its previous state.

It sounds simple concept — and it is. What’s shocking, to us anyway, is that no one has built this fundamental capability for DevOps teams. It’s a crucial piece of application management – and one we are excited to bring to our customers. We think having the ability to have the visual UI tied to its code is transformative for how we approach some of the challenges around digital transformation.

Watch the full interview discussion with Jayne Groll at Jenkins World:

If you weren’t able to connect with us at DevOps World | Jenkins World, reach out or sign up for a free Applitools account.

=====

FULL TRANSCRIPT:

Jayne: Hi everyone. I’m Jayne Groll, CEO of the DevOps Institute here at DevOps World | Jenkins World in San Francisco. And I’m really excited to be joined by James Lamberti of the Applitools. Welcome.

James: Thank you. Great for you to have us. I appreciate you doing what you’re doing here. It’s exciting.

Jayne: It’s really exciting, right? I don’t know if anybody can hear the buzz in the background but the energy is really palpable.

James: It definitely is.

Jayne: Yeah, interesting. So James, tell us a little bit about Applitools and kind of what you’re expecting out of DevOps World during the next couple of days.

James: Yeah, sure. So Applitools, for those who aren’t familiar, we essentially exist to make sure the UI is visually perfect on every device, operating system, screen, mobile, desktop and so that’s what we do. And so, we actually are pretty unique entity here at Jenkins World this year. And, you know, what do we want out of it? I mean, the theme this year is transformed, as you know. And essentially, the issue or the problem that we solve for pretty much everyone in this room is digital transformation. So, we tie in really nicely in, you know, what digital transformation, I know it’s an industry buzzword, but really what that means, in a practical sense, is, you know, the UI for most brands in the world typically exists now in 500 to 1000 variations at this very moment in time. I mean, you think about your…or just our everyday lives constantly coming across broken UI and there is no brand spared. The largest brands in the world routinely have broken UI and it’s…you know, that’s really what we’re here to transform as we can get beyond this now. We built technology at Applitools that essentially solves for that problem and ensures a visually perfect world, as the t-shirt says.

Jayne: But you know the UI is really the user experience, right?

James: Yeah.

Jayne: So, you know, the influence of the interface and how the user perceives that often shapes the perception of the product, right? So, what’s your experience? What are users really looking for in terms of their UI experience?

James: Oh, I mean, I think, fundamentally, they want it to work. I mean, honestly, like you get…we’re to a point where this transformation and the number of iterations of the UI is so overwhelming, R&D and Dev teams simply can’t get up. I know in some sense of they’ve given up and honestly, customers see that. Now, the problem with that approach is that a customer can switch on a dime. I mean, I won’t name brands right now, but I was just trying to book an airline ticket and I literally had a responsive design issue on my mobile device. I could not check out. The button…that it passed the functional tests, right, from a test automation discussion. I was clicking the button to try to buy the ticket and it literally was not working, and I needed to get it done. I just switched to another airline. So, implications are pretty clear. You can have a UI issue or a bug that just looks bad for the brand, like doesn’t cast a great impression but more importantly, you can literally break the customer experience to the point where they just switch on the dime.

So, you know, we’re in a world and a situation where we can’t just give up any more and there’s no need to give up. Like you can actually solve for this problem through visual AI, which is what we do. We’ve actually created a technology that in real time, at scale, inspects the UI, spots the difference between what should be and what is, alerts the R&D team either pre-production [inaudible 00:03:42] monitoring and allows them to fix it. And so, it’s been a huge step forward for the industry and really completes that test automation chain, if you will, right? Because the UI…the reason the UI is such a problem is it exists outside a lot of the automation frameworks we’ve built up, right? And so, we no longer have to suffer this way.

Jayne: And I think that’s awesome. And I think the example that you gave really shows that it also can affect the bottom line.

James: Oh, absolutely.

Jayne: And so, it’s not just pretty, right? It’s not superfluous. It actually does have a strong influence on…

James: Huge influence.

Jayne: …brand recognition and everything that’s associated with that, which is fantastic. So, we’re here at DevOps World, so do you guys have any announcements that you want to, kind of, sneak peek us with?

James: Yeah. I know, so I think, probably the most exciting thing, and we just announced it a couple days ago, is the world’s first UI version control system. And so, let me explain what that means. Essentially, we have integrated with GitHub and we are…for the first time ever, you can actually tie your merge, right, your branch, your fork, your merge into your GitHub Repo to the UI directly, so you’re seamlessly linking the UI to that particular code merge, and tracking that history, you know, forever. And so now, you’re in a situation where if there’s a UI issue, you can tie it back to what merge caused that issue.

Jayne: Oh, that’s awesome.

James: And you have a history of, “Okay, we made these changes, who made these changes? Why do we make these changes? What exactly did the UI look like before these changes and what branch or fork was, you know, the one we need to fall back to?” And so, that’s just, I mean, it’s actually a really extraordinary…in simple concept. But for us to have the visual tied to that code build is transformative for how we deal with this problem I was just describing around digital transformation. So, that’s a big announcement and it’s really, really exciting.

Jayne: That’s extremely exciting and again, sometimes we don’t think of the UI as part of the Repo and being able to tie that together and trackback makes a lot of sense.

James: Exactly.

Jayne: So, thank you for spending some time.

James: You’re welcome.

Jayne: Really looking forward to the rest of the conference.

James: Yeah, absolutely.

Jayne: We’ve just kicked off a little while ago and really wish you a lot of luck with your great announcement. Thanks.

James: Thank you very much. Thank you, DevOps.com.

The post How to improve Customer Experience with Visual AI and Applitools appeared first on Automated Visual Testing | Applitools.

You know what’s crazy about software?

It’s the fact that, for many businesses, applications are a big revenue driver — maybe even the ONLY revenue driver. But we have no system of record to manage the development of our application user interfaces.

Let me explain.

Suppose your company’s revenue drops off. You learn that over the past quarter, revenue coming from mobile app has declined significantly.

You dig further. App downloads are unchanged. App performance is unchanged. App crash rates are unchanged and low.

In fact, the only change is in the UI. You’ve released several new versions over the past few months. So, some of these UI changes must be causing the revenue dropoff, you guess.

You decide to do some user research. Show them old and new versions of the UI and ask which they prefer.

But no one has a record of what the old UI looked like. Those designs are lost, washed away by the passing of time like a sand castle in surf.

It’s crazy that, in 2018, we have accounting systems to track our company finances, CRM to manage sales process — heck, we even have apps to manage our houseplants!

But there’s nothing to manage the application user interfaces that drive revenue for modern businesses.

Like I said: crazy.

So how do we fix this?

Introducing UI Version Control

We were amazed that, with so many businesses driving the bulk of their revenue through web and mobile apps, there wasn’t a way for product teams to track the visual evolution of their apps. So we built one.

We call this UI Version Control. It’s like a wayback machine for your apps.

UI Version Control lets you build software smarter by capturing the entire visual history of your apps, so you have a record of what’s been changed, by whom, and when — and what should be rolled back if the need arises.

Move Fast and Fix Things

Most important, UI Version Control directly links changes in the UI to underlying code changes in your GitHub code repository. This clarifies what code commits need to be undone in order to fix a user interface change that has hurt the user experience or revenue generation.

This helps your company on a number of levels.

They can release more frequently, secure in the knowledge that they know what code changes to make to roll back a feature that didn’t pan out.

They can begin to view application features as experiments — some of which will work, and some won’t — and iterate their way to an optimal UI.

They can de-risk the user-facing aspects of their Digital Transformation projects — many of which are at risk, due in part to lack of a system of record for digital transformation — because they have a way to back out of digital dead ends.

So how do you derisk Digital Transformation?

With a UI System of Record

Before UI Version Control, Applitools showed only baselines in the context of test results from the test results manager, and you weren’t able to view the history of a test baseline.

That’s all changed.

With UI Version Control, you can see the history of all your test baselines in each branch, compare them to prior versions of your baselines, and revert to an earlier version if necessary. You can do this by selecting ‘Save as latest’ from the baseline options menu. So, if you accidentally accepted a baseline that you shouldn’t have, you can undo your mistake.

UI Version Control works just like source code version control that you, or your developers, are already familiar with. Visual baseline branches can be merged in the same way that code changes are merged.

For each branch in your GitHub project repository, you can see the history of all your test baselines, compare them to prior baseline versions, and if necessary, revert to an earlier version. You can then reject and/or revert to any baseline modified by accident or by design.

Here’s a demonstration from our CTO, Adam Carmi, of UI Version Control:

Conclusion

Applitools UI Version Control is now immediately available to all Applitools customers. So, while we might not be able to prevent you from building bad UIs, at least we’ll help you undo them faster.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

What recent UI changes to your app are you worried about? 

The post How to improve your user interfaces with UI Version Control appeared first on Automated Visual Testing | Applitools.

While everyone knows that printed newspaper sales are dwindling, the interesting story is in online subscriptions, which are exploding. Electronic Medical Records (EMR) are becoming the new normal, eliminating lookup, replacement and a whole lot of printing. The Plain Old Telephone system (POTS), which phone poles and a separate physical wire into the house are falling into disrepair but anyone under 30 is unlikely to notice because they do not have a physical landline phone.

The digital transformation is in full swing and the consequences are everywhere.

Can software testers help smooth digital transformations with visual UI testing?

I think so.

Quick Response to Customer Needs

Several years ago I was working on a software product used by nurses and anaesthesiologist. Our software was meant to be a bridge between the world of paper charting and a completely paperless workflow. Our initial offering was a digital pen solution. Doctors would use a special electronic pen to write on forms that looked no different from what they used daily for years. At the end of the day, a doctor or nurse syncs the pen to a computer using a USB cable, and all of those forms were magically transferred to our dashboard. The flow from documenting with a pen to having digital patient information was supposed to be as close to what the doctors normally do as possible.

The next stage in this product was something completely new, an iPad app.

The further a software company moves from a workflow people have been using for the last 20 years, the more problems they will encounter. Starting a transformation with a visual UI testing strategy can help software makers respond to customer needs before the customers realize they have them. Visual testing tools provide instant feedback on the look and feel of your product with every build. Rather than building a new feature and waiting for the market to give feedback you can build a prototype, get user experience feedback from a customer champion, and make that your visual baseline.

Cross Platform Delivery

Before a digital transformation, companies might deal in one medium. For the doctors we were supporting, that was pen and paper. This product ran in a few versions of Internet Explorer, as well as 5 or 6 mobile platforms. Using visual testing, we could have had a better feel for consistency across platforms and devices.

A visual UI testing strategy simplifies the cross-platform testing problem. Start with a complete scan of the elements on each page of the browser you care about the most. This baseline gives you a reference point for every other place you need to test, and also is a way to explore the full feature set in your product. Once you have that baseline, you can run your visual UI testing suite in Continuous Integration and discover how each platform and operating system – iOS, Android, iPad, iPad mini, Google Tablets, and a variety of mobile phones – are different. Your development team will discover that an iPhone 6 running iOS 8 is missing a submit button that displays perfectly everywhere else before your customers.

Dealing with Humans

Software customers tend to be change averse. Take something that might be clunky, but has worked for them for the past 20 years, and turn it into software. What usually happens here is shelf-ware, products that get no use because people either couldn’t figure out how to use them or the new software was more painful than what they had before. This pain can come from software bugs, it can come from bad design, or it can come from normal ways of working before being completely absent from the new software solution. Consistency is one of the most important factors for a successful digital transformation. A visual UI testing strategy will give your development team an early alert on how your products are different between browsers and platforms. That means an easier digital transformation for your customers.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Visual UI Testing for Digital Transformation appeared first on Automated Visual Testing | Applitools.

Watch Joe Colantonio interview test automation executive Adam Auerbach, as he takes a deep dive into the ecosystem of continuous delivery and digital transformation, including: challenges, skills, tools, and best practices.

Adam Auerbach – a seasoned executive in leading financial institutions – shared with us his first-hand knowledge and experience about the growing pains Dev/Test and DevOps teams face on their journey to Continuous Delivery, and the challenges they need to overcome in order to deliver better and faster in the era of digital transformation.

Adam also discussed how culture, processes, technical infrastructure, tools, and people need to adapt to this new and rapidly-changing ecosystem, as well as acquire and master new best practices for delivering quality and speed.

Key talking points included:

• DevOps in the real world: why and how DevOps shine in a Continuous Delivery ecosystem
• Staying relevant: changing roles and responsibilities of Dev/Test
• Evolution of software testing: skill sets and best practices required to survive the age of digital transformation
• Risk mitigation: how to do it properly with continuous testing and test automation
• Supporting CI-CD culture: How to properly create and maintain culture and processes on the journey to Continuous Delivery
• Nuts and bolts: technical infrastructure and tools required for real-time test automation and Continuous Delivery excellence

Watch it now:

And find Joe’s slide deck below:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Putting Quality First: Evolution of Dev/Test and DevOps in the Era of Continuous Delivery appeared first on Automated Visual Testing | Applitools.

These days, to make an impact on your customers and reach new audiences, it’s not enough to have a great product. A company without a strong digital presence – and by extension, a great digital experience – can fall behind.

What do we mean by the digital experience? This translates to delivering an app that works functionally and looks beautiful across any digital platform that your customers care about. 

But creating this experience isn’t always easy. It’s hard to keep up with an ever-growing number of different devices, browsers and screen resolutions.

Adjusting your site or app to the huge variety of platforms out there is a time-consuming task, and you might find yourself asking how to maintain a great digital experience without wasting time on QA, physical device testing and other maintenance procedures.

However, maintenance and testing is key to delivering a good digital experience. Data collected from users’ online behaviour teaches us that in many cases, a user who has closed your app after encountering a bug won’t open it up again.
Moreover, a visual-faulty UI creates an unprofessional feel to your site or app and a negative reputation to your brand. To keep your customers engaged and to ensure they are loyal and excited about your site or app, you must provide them a friendly and flawless experience.

With today’s growing numbers of operating systems, devices and browsers, a flawless digital experience can be achieved only by automatically detecting UI bugs and monitoring your site or app, assuring continuous quality in your product’s lifecycle. Assuring a visually perfect experience across multiple platforms requires the automation of your testing – not only of the functional aspects of your site or app, but the visual ones as well.

Hello, Automated Visual Testing

This is why the practice of Automated Visual Testing is gaining huge momentum right now. Whether you’re a Digital Experience Executive or QA Manager, Automated Visual Testing can offer you many benefits.

Visual Testing allows you to handle changes in your site or app that would have otherwise demanded a large amount of work using conventional testing methods by your developers and QA teams.
Both functional and visual bugs in your UI that would have taken hours to detect using manual QA personnel can now be automatically detected. This allows your development team to work better and faster, while widening your feature scope and shortening your release cycle. The automation of your visual tests will allow your developers and QA engineers to push your product forward, instead of chasing and fixing UI bugs.

Adding Automated Visual Testing to your existing testing infrastructure doesn’t take much effort. Implementation into your workflow will take some time, but automating your visual tests will quickly boost your team’s productivity, enable you to test and monitor your digital experience across all platforms.

Keep your customers happy – give them a flawless experience by automating your visual tests today!

Contact us to learn more about how Automated Visual Testing can help you ensure a flawless digital user experience.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Why Digital Executives Should Care about Automated Visual Testing appeared first on Automated Visual Testing | Applitools.