In this blog post we introduce a solution to deal with data persistence. We developed it for the Just Eat iOS app and we call it JustPersist. It’s available open source on Github at github.com/justeat/JustPersist.

JustPersist aims to be the easiest and safest way to do persistence on iOS with Core Data support out of the box. It also allows you to migrate to any new persistence framework with minimal effort.

The main author behind JustPersist and its design is Keith Moon. Major kudos to Keith for the excellent execution in Swift!

Overview

At Just Eat, we persist a variety of data in the iOS app. In 2014 we decided to use MagicalRecord as a wrapper on top of Core Data but over time the numerous problems and fundamental thread-safety issues, arose. In 2017, MagicalRecord is not supported anymore and new solutions look more appealing. We decided to adopt Skopelos: a much younger and lightweight Core Data stack, with a simpler design, developed by Alberto De Bortoli, one of our engineers. The design of the persistence layer interface gets inspiration from Skopelos as well, and we invite the reader to take a look at its documentation.

The main problem in adopting a new persistence solution is migrating to it. It is rarely easy, especially if the legacy codebase doesn’t hide the adopted framework (in our case MagicalRecord) but rather spread it around in view controllers, managers, helper classes, categories and sometimes views. Ultimately, in the case of Core Data, there is a single persistent store and this is enough to make impossible to move access across “one at a time”. There can only be one active persistence solution at a time.

We believe this is a very common problem, especially in the mobile world. We created JustPersist for this precise reason and to ease the migration process.

At the end of the day, JustPersist is two things:

• a persistence layer with a clear and simple interface for transactional readings and writings (Skopelos-style)
• a solution to migrate from one persistence layer to another with (we believe) the minimum possible effort

JustPersist aims to be the easiest and safest way to do persistence on iOS. It supports Core Data out of the box and can be extended to transparently support other frameworks. Since moving from MagicalRecord to Skopelos, we provide available wrappers for these two frameworks.

The tone of JustPersist is very much Core Data-oriented but it enables you to migrate to any other persistence framework if a custom data store (wrapper) is implemented (in-memory, key-value store, even Realm if you are brave enough).

JustPersist is available through CocoaPods. To install it, simply add the following line to your Podfile:

Using only pod JustPersist will add the core pod with no subspecs and you’ll have to implement your own wrapper to use the it. If you intend to extend JustPersist to support other frameworks, we suggest creating a subspec.

Usage of the persistence layer

To perform operation you need a data store, which you can setup like this (or see “common way of setting up a data store”):

Before using the data store for the first time, you must call setup() on it, and possibly tearDown() when you are completely done with it.

We suggest setting up the stack at app startup time, in the applicationDidFinishLaunchingWithOptions method in the AppDelegate and to tear it down at the end of the life cycle of your entire app, when resetting the state of the app (if you provide support to do so) or in the tearDown method of your unit tests suite.

To hide the underlying persistence framework used, JustPersist provides things that conform to DataStoreItem and MutableDataStoreItem, rather than the CoreData specific NSManagedObject. These protocols provide access to properties using objectForKey and setObject:forKey: methods.

In the case of Core Data, JustPersist provides an extension to NSManagedObject to make it conforming to MutableDataStoreItem.

Readings and writings

The separation between readings and writings is the foundation of JustPersist.
Reading are always synchronous by design:

While writings can be both synchronous or asynchronous:

The accessor provided by the blocks can be a read one (DataStoreReadAccessor) or a read/write one (DataStoreReadWriteAccessor). Read accessors allow you to do read operations such as:

While the read/write ones allow you to perform a complete set of CRUD operations:

To perform an operation you might need a DataStoreRequest which can be customized with itemType, an NSPredicate, an array of NSSortDescriptor, offset and limit. Think of it as the corresponding Core Data’s NSFetchRequest.

Here are some complete examples:

In write blocks there is no need to make any call to a save method. Since it would be the obvious thing to do at the end of a transactional block, JustPersist does it for you. Read blocks are not meant to modify the store and you wouldn’t even have the API available to do so (unless DataStoreItem objects are casted to NSManagedObject in the case of CoreData to allow the setting of properties), therefore a save will not be performed under the hood.

Common way of setting up a data store

We recommend to use dependency injection to pass the data store around but sometimes it might be hard. If you wish to access your data store via a singleton, here is how your app could create a shared instance for the DataStoreClient (e.g. DataStoreClient.swift) using Skopelos.

For unit tests, you might want to use the inMemoryShared for better performance.

Child data store

A child data store is useful in situations where you might have the need to rollback all the changes performed in a specific section of the app or in a part of the user journey. Think of it as a scratch/disposable context in the Core Data stack by Marcus Zarra.

At Just Eat we use a child data store for the addition of complex products to the basket. The user might make many updates to the product and it is easier to perform the final save operation when the user confirms the addition rather than dealing with multiple CRUD operations on the main data store.

A child data store behaves just like a normal data store, with the only exception that, to save the changes back to the main data store, developers must explicitly merge the data stores. Here is a complete example:

Thread-safety notes

Read and sync write blocks are always performed on the main thread, no matter which thread calls them.
Async write blocks are always performed on a background thread.

Sync writings return only when the changes are persisted (in the case of Core Data, usually to the NSManagedObjectContext with main concurrency type).

Async writings return immediately and leave the job of saving to the source of truth to JustPersist (whether it be the context or a persistent store). They are eventual consistent, meaning that the next reading could potentially not have the data available.

Forcing a transactional programming model for readings and writings helps developers to avoid thread-safety issues which in Core Data can be caught setting the -com.apple.CoreData.ConcurrencyDebug 1 flag in your scheme (which we recommend enabling).

How to migrate to a different persistence layer

Examples in this sections are in Objective-C as 1. they deal with the legacy code for the nature of the example and 2. to show that JustPersist works just fine with Objective-C too.

Here we’ll outline the steps we made to migrate away from MagicalRecord to Skopelos using JustPersist. We believe that a lot of apps still use MagicalRecord, so this may apply to your case too. If your need is to move from and to other 2 frameworks, you need to implement the corresponding data stores to wrap them.

You should start by implementing your DataStoreClient (you could follow the steps in “common way of setting up a data store” and allocating the data store for the current persistence layer used by your app in the sqliteStack method and possibly in the inMemoryStack one too. In our case, since we want to move away from MagicalRecord, the data store used would be MagicalRecordDataStore.

Standard CRUD interactions with MagicalRecord are like so:

All of them should be converted one by one to JustPersist:

You should make sure you don’t perform any UI work within the blocks even if the read and writeSync ones are executed on the main thread. Actually, you should aim for doing only the necessary work related to interact with the persistence layer, which often might be copying values out of objects to have them accessible outside the block (in Objective-C via the __block keyword). Developers should not hold references to model objects to pass them around threads (transactional blocks help ensure such rule).

By having moved all the direct interactions from MagicalRecord to JustPersist, you should be now able to remove all the various @import MagicalRecord and #import <MagicalRecord/MagicalRecord.h> from the entire codebase.

Once At this point, your DataStoreClient can be modified to allocate the target data store in the sqliteStack and inMemoryStack methods. In our case, the SkopelosDataStore.

Conclusion

JustPersist aims to be the easiest and safest way to do persistence on iOS. It supports Core Data out of the box and can be extended to transparently support other frameworks.

You can use JustPersist to migrate from one persistence layer to another with minimal effort. Since we moved from MagicalRecord to Skopelos, we provide available wrappers for these two frameworks.

At its core, JustPersist is a persistence layer with a clear and simple interface to do transactional readings and writings, taking inspirations from Skopelos where readings and writings are separated by design.

We hope this library will ease the process of setting up a persistence stack, avoiding the common headache of Core Data and potential threading pitfalls.

About the authors

Alberto De Bortoli is the Principal iOS Engineer at Just Eat.
Keith Moon is a Senior iOS Engineer in the Payments team at Just Eat.

Overview

At Just Eat, tracking events is a fundamental part of our business analysis and the information we collect informs our technical and strategic decisions. To collect the information required we needed a flexible, future-proof and easy to use tracking system that enables us to add, remove and swap the underlying integrations with analytical systems and services with minimal impact on our applications’ code. We also wanted to solve the problem of keeping the required event metadata up-to-date whenever the requirements change.

JustTrack is the event tracking solution we built for that and it’s available open source on Github at //github.com/justeat/JustTrack.

Examples and documentation are available in the Github repository Readme.

Main features

• Events are declared in a .plist file and Swift 3 code is automatically generated at build time from it.
• Events can be sent to multiple destinations (called Trackers) at the same time.
• Custom Trackers are easy to create and use.

Events Definition

One of the problems we found with existing solutions is that the events are declared in code and therefore can only be maintained by developers. Similarly, existing solutions offer very generic tracking facilities for developers. Because of that, whenever the required metadata associated with an event changes for any reason, the developer has to search the code base and update all instances of the event with the correct implementation. This, of course, is a very fragile process and is prone to errors.

JustTrack solves these problems by declaring events in a plist file that is used to automatically generate equivalent definitions of the events in Swift that can be used in the app. This provides several benefits:

• Each event is uniquely identified.
• The metadata associated with each event is type checked.
• When the requirements for an event change, the developers can see it through build errors and warnings that will naturally occur.
• Plists can be edited as XML, which means anybody in the business can edit them.
• It’s easy to search for events that are no longer used and deleted events won’t compile.

An Event is made of:

• Name: the unique identifier
• Registered Trackers: List of event destinations (e.g. Google Analytics)
• Payload: The metadata associated with the event (at this time only String key-value pairs are supported)

Trackers

Another problem we found with existing solutions is that, generally speaking, all the events need to be tracked with the same Trackers. The developer doesn’t have any freedom to decide which event goes to which Tracker, and several solutions only support specific tracking technologies (such as GA, Firebase, and so on)

JustTrack solves this problem by allowing the developer to specify the “registered” trackers for each event and to create custom trackers.

A Tracker is an object implementing the JETracker protocol and is loaded using: tracker.loadCustomTracker( ... ) function. You can implement any tracker you want and JustTrack provides a few default trackers:

• JETrackerConsole – print events to the system’s console
• JEFacebookTraker (not yet implemented)
• JEGoogleAnalyticsTraker (not yet implemented, Google’s pods can’t be used as a dependency in a pod)
• JETrakerFirebase (not yet implemented, Google’s pods can’t be used as a dependency in a pod)

Conclusions

Using JustTrack you can easily track your app’s events to multiple destinations, having all the necessaire flexibility, intrinsic events documentation and all the expandability you will ever need in future, all of that writing the minimum amount of code possible.

About the author

Federico Cappelli is a Senior iOS Engineer and iOS COG Leader at Just Eat.

In this blog post we introduce the solution for local and remote logging we developed for the Just Eat iOS app. It’s named JustLog and it’s available open source on Github at github.com/justeat/JustLog.

Overview

At Just Eat, logging and monitoring are fundamental parts of our job as engineers. Whether you are a back-end engineer or a front-end one, you’ll often find yourself in the situation where understanding how your software behaves in production is important, if not critical. The ELK stack for real-time logging has gained great adoption over recent years, mainly in the back-end world where multiple microservices often interact with each other.

In the mobile world, the common approach to investigating issues is gathering logs from devices or trying to reproduce the issue by following a sequence of reported steps. Mobile developers are mostly familiar with tools such as Google Analytics or Fabric.io but they are tracking systems, not fully fledged logging solutions.

We believe tracking is different in nature from logging and that mobile apps should take advantage of ELK too in order to take their monitoring and analysis to another level. Remote logging the right set of information could provide valuable information that would be difficult to gather otherwise, unveil unexpected behaviors and bugs, and even if the data was properly anonymized, identify the sequences of actions of singular users.

JustLog takes logging on iOS to the next level. It supports console, file and remote Logstash logging via TCP socket out of the box. You can also setup JustLog to use logz.io with no effort. JustLog relies on CocoaAsyncSocket and SwiftyBeaver, exposes a simple swifty API but it also plays just fine with Objective-C.

JustLog sets the focus on remote logging, but fully covers the basic needs of local console and file logging.

Usage

JustLog, is available through CocoaPods. To install it, simply add the following line to your Podfile:

Import it into your files like so:

This logging system strongly relies on SwiftyBeaver.
We decided to adopt SwiftyBeaver due to the following reasons:

• good and extensible design
• ability to upload logs to the cloud
• macOS app to analyze logs

A log can be of one of 5 different types, to be used according to the specific need. A reasonable adopted convention on mobile could be the following:

• 📣 verbose: Use to trace the code, trying to find one part of a function specifically, sort of debuggin with extensive information.
• 📝 debug: Information that is diagnostically helpful to developers to diagnose an issue.
• ℹ️ info: Generally useful information to log (service start/stop, configuration assumptions, etc). Info to always have available but usually don’t care about under normal circumstances. Out-of-the-box config level.
• ⚠️ warning: Anything that can potentially cause application oddities but an automatic recovery is possible (such as retrying an operation, missing data, etc.)
• ☠️ error: Any error which is fatal to the operation, but not the service or application (can’t open a required file, missing data, etc.). These errors will force user intervention. These are usually reserved for failed API calls, missing services, etc.

When using JustLog, the only object to interact with is the shared instance of the Logger class, which supports 3 destinations:

• sync writing to Console (custom destination)
• sync writing to File (custom destination)
• async sending logs to Logstash (usually part of an ELK stack)

Following is a code sample to configure and setup the Logger. It should be done at app startup time, in the applicationDidFinishLaunchingWithOptions method in the AppDelegate.

The defaultUserInfo dictionary contains a set of basic information to add to every log.

The Logger class exposes 5 functions for the different types of logs. The only required parameter is the message, optional error and userInfo can be provided. Here are some examples of sending logs to JustLog:

It plays nicely with Objective-C too:

The message is the only required argument for each log type, while userInfo and error are optional.
The Logger unifies the information from message, error, error.userInfo, userInfo, defaultUserInfo and call-site info/metadata in a single dictionary with the following schema form of type [String : Any] (we call this ‘aggregated form’). E.g. in JSON representation:

All destinations (console, file, logstash) are enabled by default but they can be disabled at configuration time like so:

The above 5 logs are treated and showed differently on the each destination:

Console

The console prints only the message.

File

On file we store all the log info in the ‘aggregated form’.

Logstash

Before sending a log to Logstash, the ‘aggregated form’ is flattened to a simpler [String : Any] dictionary, easily understood by Logstash and handy to be displayed on Kibana. E.g. in JSON representation:

Which would be shown in Kibana as follows:

A note on Logstash destination

The logstash destination is configured via properties exposed by the Logger. E.g.:

When the logLogstashSocketActivity is set to true, socket activity is printed to the console:

This destination is the only asynchronous destination that comes with JustLog. This means that logs to Logstash are batched and sent at some point in future when the timer fires. The logstashTimeout property can be set to the number of seconds for the dispatch.
In some cases, it might be important to dispatch the logs immediately after an event occurs like so:

or, more generally, in the applicationDidEnterBackground and applicationWillTerminate methods in the AppDelegate like so:

Sending logs to logz.io

JustLog supports sending logs to logz.io.

At the time of writing, logz.io uses the following host and port (please refer to the official documentation):

When configuring the Logger (before calling setup()), simply set the token like so:

Conclusion

JustLog aims to be an easy-to-use working solution with minimal setup. It covers the most basic logging needs (console and file logging) via the great foundations given by SwiftBeaver, but also provides an advanced remote logging solution for Logstash (which is usually paired with Elasticsearch and Kibana in an ELK stack). JustLog integrates with logz.io, one of the most widely used ELK SaaS, placing itself as the only solution in the market (at the time of writing) to leverage such stack on iOS.

We hope this library will ease the process of setting up the logging for your team and help you find solutions to the issues you didn’t know you had.

About the author

Alberto De Bortoli is the Principal iOS Engineer at Just Eat.

An iOS framework that ports Force-Touch Peek/Pop-like interactions to devices that aren’t force-touch enabled

At Just Eat, teams tend to build software with the community in mind. Over time, Just Eat has published several Open Source projects and has been an important pillar to the C# community.

More recently, the Company has grown its iOS team in London to focus on improving the Just Eat app for the UK market. With this growth phase, I had the opportunity to join the team for some time.

As part of the effort to improve our app, launch the new Just Eat brand and build modules to help us in the long run, we’re building and open sourcing more iOS projects than ever before. JustPeek, is the first of them to get published.

JustPeek is an iOS library that adds support for Force Touch-like Peek / Pop interactions on devices that don’t natively support it due to lack of force recognition capability in the screen. Under the hood it uses the native implementation if available, otherwise a custom implementation based on UILongPressGestureRecognizer. It fallbacks to the latter also on the iOS Simulator if needed.

Unlike similar libraries already available on the web, JustPeek tries to mimic entirely the original implementation. It doesn’t simply use screenshots to display a preview, but previews the UIViewController itself. As of version 0.2.0, it also supports committing to a preview in the fallback implementation. As there’s no way to measure pressure on old devices, committing happens when a user keeps the preview open for more than a certain amount of time – 3 seconds at the time of writing.

Here’s how you use JustPeek:

Here’s what it looks like in the demo application we ship with the code, as run on the iOS Simulator:

We use JustPeek in the Just Eat UK app, in the list of results you get when searching for restaurants that serve your area, to show a preview of popular dishes a restaurant has to offer.

JustPeek is now publicly available on GitHub and can be added to your projects using CocoaPods. We really hope you’ll like it!

About the author

Gianluca Tranchedone is a Senior iOS Engineer and, at the time of writing, the main contributor of JustPeek.

Some time in the future, the age of the smartphone will draw to a close and experiences will become more in-tune with the way humans actually live. We need to be thinking about this new wave of interactions at a time when our customer’s attention is a premium. We need to be augmenting their worlds, not trying to replace them…

I’m Craig Pugsley – a Principal UX Designer in Product Research. Our team’s job is to bring JUST EAT’s world-leading food ordering experience to the places our consumers will be spending their future, using technology that won’t be mainstream for twelve to eighteen months.

It’s a great job – I get to scratch my tech-geek itch every day. Exploring this future-facing tech makes me realise how old the systems and platforms we’re using right now actually are. Sometimes it feels like we’ve become their slaves, contorting the way we want to get something done to match the limitations of their platforms and the narrow worldview of the experiences we’ve designed for them. I think it’s time for change. I think smartphones are dumb… I feel like we’ve been led to believe that ever more capable cameras or better-than-the-eye-can-tell displays make our phones more useful. For the most part, this is marketing nonsense. For the last few years, major smartphone hardware has stagnated – the occasional speed bump here, the odd fingerprint sensor there… But nothing that genuinely makes our phones any smarter. It’s probably fair to say that we’ve reached peak phone hardware.

What we need is a sea-change. Something that gives us real value. Something that recognises we’re probably done with pushing hardware towards ever-more incremental improvements and focuses on something else. Now is the time to get radical with the software.

I was watching some old Steve Jobs presentation videos recently (best not to ask) and came across the seminal launch of the first iPhone. At tech presentation school, this Keynote will be shown in class 101. Apart from general ambient levels of epicness, the one thing that struck me was how Steve referred to the iPhone’s screen as being infinitely malleable to the need – we’re entirely oblivious to it now, but at that time phones came with hardware keyboards. Rows of little buttons with fixed locations and fixed functions. If you shipped the phone but thought of an amazing idea six months down the line, you were screwed.

In his unveiling of the second generation of iPhone, Jobs sells it as being the most malleable phone ever made. “Look!” (he says), “We’ve got all the room on this screen to put whatever buttons you want! Every app can show the buttons that make sense to what you want to do!”. Steve describes a world where we can essentially morph the functionality of a device purely through software.

But we’ve not been doing that. Our software platforms have stagnated like our hardware has. Arguably, Android has basic usability issues that it’s still struggling with; only recently have the worse Bloatware offenders stopped totally crippling devices out-the-box. iOS’s icon-based interface hasn’t changed since it came out. Sure, more stuff has been added, but we’re tinkering with the edges – just like we’ve been doing with the hardware. We need something radically different.

One of the biggest problems I find with our current mobile operating systems is that they’re ignorant of the ecosystem they live within. With our apps, we’ve created these odd little spaces, completely oblivious to each other. We force you to come out of one and go in the front door of the next. We force you to think first not about what you want to do, but about the tool you want to use to do it. We’ve created beautiful rooms.

Turning on a smartphone forces you to confront the rows and rows of shiny front doors. “Isn’t our little room lovely” (they cry!) “Look, we’ve decorated everything to look like our brand. Our tables and chairs are lovely and soft. Please come this way, take a seat and press these buttons. Behold our content! I think you’ll find you can’t get this anywhere else… Hey! Don’t leave! Come back!”

“Hello madame. It’s great to see you, come right this way. Banking, you say? You’re in safe hands with us. Please take a seat and use this little pen on a string…”

With a recent iOS update, you’re now allowed you to take a piece of content from one room and push it through a little tube into the room next door.

Crippled by the paralysis of not alienating their existing customers, Android and iOS have stagnated. Interestingly, other vendors have made tantalizing movements away from this beautiful-room paradigm into something far more interesting. One of my favorite operating systems of all time, WebOS, was shipped with the first Palm Pre.

There was so much to love about both the hardware and software for this phone. It’s one of the tragedies of modern mobile computing that Palm weren’t able to make more of this platform. At the core, the operating system did one central thing really, really well – your services were integrated at a system level. Email, Facebook, Twitter, Flickr, Skype, contacts – all managed by the system in one place. This meant you could use Facebook photos in an email. Make a phone call using Skype to one of your contacts on Yahoo. You still had to think about what beautiful room you needed to go into to find the tools you needed, but now the rooms were more like department stores – clusters of functionality that essentially lived in the same space.

Microsoft took this idea even further with Windows Phone. The start screen on a Windows Phone is a thing of beauty – entirely personal to you, surfacing relevant information, aware of both context and utility. Email not as important to you as Snapchat? No worries, just make the email tile smaller and it’ll report just the number of emails you haven’t seen. Live and die by Twitter? Make the tile huge and it’ll surface messages or retweets directly in the tile itself. Ambient. Aware. Useful.

Sadly, both these operating systems have tiny market shares.

But the one concept they both share is a unification of content. A deliberate, systematic and well executed breaking down of the beautiful room syndrome. They didn’t, however, go quite far enough. For example, in the case of Windows Phone, if I want to contact someone I still need to think about how I’m going to do it. Going into the ‘People Hub’ shows me people (rather than the tools to contact them), but is integrated only with the phone, SMS and email. What happens when the next trendy new communication app comes along and the People Hub isn’t updated to support the new app? Tantalizingly close, but still no cigar.

What we need is a truly open platform. Agnostic of vendors and representing services by their fundamentally useful components. We need a way to easily swap out service providers at any time. In fact, the user shouldn’t know or care. Expose them to the things they want to do (be reminded of an event, send a picture to mum, look up a country’s flag, order tonight’s dinner) and figure out how that’s done automatically. That’s the way around it should be. That’s the way we should be thinking when designing the experiences of the future.

Consider Microsoft’s Hololens, which was recently released to developers outside of Microsoft. We can anticipate an explosion of inventiveness in the experiences created – the Hololens being a unique device leapfrogging the problem of beautiful rooms to augment your existing real-world beautiful rooms with the virtual.

Holographic interface creators will be forced to take into account the ergonomics of your physical world and work harmoniously, contextually, thoughtfully and sparingly within it. Many digital experience designers working today should admit to the fact that they rarely take into account what their users were doing just before or just after their app. This forces users to break their flow and adapt their behavior to match the expectations of the app. As users, we’ve become pretty good at rapid task switching, but doing so takes attention and energy away from what’s really important – the real world and the problems we want to solve.

Microsoft may be one of the first to market with Hololens, but VR and AR hardware is coming fast from the likes of HTC, Steam, Facebook and Sony. Two-dimensional interfaces are on the path to extinction, a singular event that can’t come quick enough.