Zebra: Full editing with Neos and Next.js

Note: This post is based on my talk from the Neos Conference 2023. It goes into a little bit more depth as our usual blog posts - a more TL;DR version of this will follow.

The Future™

The way we create for the web and the web itself are constantly changing. New approaches complement - or replace - established practices.

There are many new paradigms for creating web experiences. From a web site focused on content to a complex application. Common to them is the use of frontend technologies on the server and a more streamlined build process - where build tooling is part of the framework. You might already know some of these and there are many more. This is just a selection of the most popular hybrid site generators.

We currently focus on Next.js, but our approach is not limited to it.

Why use it?

Before we dive deeper into new things, it's always a good question to ask "why". Why should we use it?

There is of course not one single answer, but as we will see, we open up a world of new possibilities with these tools.

A website / e-commerce example

Let's have a look at an example of a website with some e-commerce features to see how crafting the web experience changes with Next.js.

If we break down the structure of a website, we can identify a few parts:

• A logo and navigation in the header
• A cart widget which shows the number of items in the cart
• Product details with an image, content and an "add to cart" button

Some of them are pretty static, others have to fetch content, like a product image and description. And other parts are interactive, like adding the product to the cart and showing the number of items in the cart.

In a more classic server-side CMS architecture, we would generate the markup via the CMS and deliver the code for dynamic features via JavaScript, that was built through some kind of build tooling.

There are lots of implementation details to figure out that need to cross various layers:

• Calls to the shop features could go to a headless shop API and both the frontend and the CMS would have to connect to it.
• Implementing a more complex feature often involves changes inside the CMS and to the JavaScript part. And maybe even adding some custom routes for fetch requests. So it's not a very coherent experience and a mix of different technologies need to be used.

With Next.js, we have more integration at the boundary between client and server:

• Everything is a React component, there is no architectural difference between static, dynamic or interactive parts.
• And most important to note is, that requests from the browser go to Next.js and will generate the markup of the site through server-side rendering. This is also where data fetching typically happens.
• After the browser loaded the site the same React components can work client-side for interactive experiences.
• The CMS is now also headless and just one of the data sources of the site.
• We can now implement features across the client and server layers much easier, and Next.js also builds all of our frontend assets and takes care of things like code splitting and other optimizations.

But what are these different modes of rendering actually?

These types of rendering give a lot of flexibility and we can get rid of things like caching and other special publishing solutions.

How can we use it with Neos?

Now that we know, why we want to use this type of architecture - how does Neos fit into it? Let's go through our journey together to understand what the choices and obstacles were.

First goal: Use Neos CMS for content in Next.js

Our first goal was to use Neos as the source for content in a Next.js frontend. That means we would like to have all document nodes in Neos available as pages in Next.js. We also need to implement the rendering for all content node types that we are using.

One of our first big questions was, whether we want to go fully headless or still have some kind of rendering defined in Neos:

If we were to go headless, there would be no Fusion for rendering in Neos. All markup would be created via React components. And there would have to be a mapping from all node types to components and their props.

As a benefit, we could move the frontend aspects out of Neos - that means no Fusion code that generates HTML and also all asset like CSS and JavaScript would go into Next.js.

But at first we were reluctant to implement this detailed integration. So we considered another hybrid approach, where Neos provides some structured data about our documents and nodes via an API, but markup will be rendered via Fusion to save all the detailed effort.

After a first brief attempt, it quickly became clear that this would lead to a frontend in Next.js with a manageable effort. But - and this is even more important - the architecture would be further mixed and we would miss out on a lot of opportunities we would get from a more ideal approach.

In the end you would miss out on a lot of opportunities, if there's still some rendering defined in Neos, so we went for the headless approach.

Content API

During this time of experimentation we created the Content API package to provide a simple JSON API to fetch a list of documents - for routing - and a detail view for each document node:

The actual data that is returned for each document node is declared via Fusion - that makes it extensible and we first experimented with the hybrid approach by mixing JSON and HTML.

But the same package could also be used to have a more generic output of just the node properties without any rendering in Neos - for the headless solution.

So how do we use it to have all the rendering in Next.js?

We add a catch all route for all document route paths to Next.js by naming the file pages/[[...path]].ts.

• It's just a convention and the Next.js documentation has more details about this.
• path refers here to the parameter name where we can access the route path.

So what will happen if we build the Next.js application?

• Next.js fetches all documents via getStaticPaths to know about all route paths from the content API.
• Data for the content will be fetched from the API for each document via getStaticProps.

These steps are performed to render the actual frontend in Next.js:

1. By supplying a "path" we fetch all the data that we need to render the page in Next.js from the content API.
2. Each page is rendered by a React component using the data for the node itself including child nodes, the site and other meta information like navigation or footer items.
3. The actual React component for a node type is resolved from a simple mapping. And It's a good idea to split components for content and presentational purposes for better re-use.
4. A React context is used to pass the document data and props of the current node down the component tree.

Code in Neos

This is basically all the Fusion code we need in Neos to declare the data that is returned from the content API:

prototype(Zebra.Site:Document.Page.Api) <
  prototype(Networkteam.Neos.ContentApi:DefaultDocument) {

  meta {
    isRootPage = ${documentNode == site}

    mainNavigation = Networkteam.Neos.ContentApi:MenuItems {
      maximumLevels = 1
    }
  }
  site = Networkteam.Neos.ContentApi:BaseNode {
    @context.node = ${site}
  }

  # Render node with children recursively
  node = Networkteam.Neos.ContentApi:RecursiveNode
}

Each document node type gets its own definition, so the response can differ depending on the node type of a document. As you can see, there are no details about the actual frontend-rendering. One big advantage is, that we don't need to consider cache configuration, since all requests to the content API are performed uncached.

Code in Next.js

And these are some examples of how the actual code in Next.js looks like:

const DocumentPage = () => {
  return (
    <Layout>
      <ContentCollection nodeName="main" />
    </Layout>
  );
};

export default DocumentPage;

const ContentHeadline = () => {
  const node = useNode();
  return (
    <div className={baseClasses(node)}>
      <Headline as={node.properties.hierarchy} size={node.properties.size}>
        {node.properties.title}
      </Headline>
    </div>
  );
};

With that in place we can now generate the complete site in Next.js and need no frontend in Neos and completed our first goal - by going headless.

But what about editing?

How can we provide a familiar experience for editors to change content?

We could simply use Neos with structured editing - without a frontend preview. It could work, but that means we would loose one of the main selling points of Neos - the Neos UI with inline editing capabilities directly in the frontend.

Another thing is updating our statically generated pages if content in Neos was published - we'll come back to this later, as there are some interesting details here, and it wasn't particularly easy to get this right.

Next goal: Great editing experience

So this might be the most important idea here: what if we could use our Next.js frontend inside the UI and just pretend it was rendered by Neos?

Let's have a look on how it can be performed:

• The backend content module in Neos will request the preview route from Next.js and supply the same information as for the normal editing preview.
• This route will fetch the details from the content API via server-side rendering and forwards the Neos session for authenticated access.
• The page rendering in Next.js also needs to output additional metadata for the Neos UI.

Can we generate the exact markup for Neos UI?

Which brings us to the question whether we can generate the needed markup and notify the Neos UI after the preview page was loaded.

• This took quite some reverse-engineering and trial-and-error to figure out all the needed bits and pieces.
• I will spare you all the details at this point, but as you might have guessed it is indeed possible to use all the Neos UI features with a pure Next.js frontend.
• One central idea was to use hooks inside the content components to generate the attributes and serialized node data if we are rendering for the backend.

We also inject the required CSS and JavaScript for the Neos UI by returning it in a structured way from the content API.

An editable content component

Have a look at this example to see how a React component with editing support would look like:

const ContentHeadline = () => {
  const node = useNode();
  return (
    <ContentComponent className={baseClasses(node)}>
      <Headline as={node.properties.hierarchy} size={node.properties.size}>
        <Editable property="title" />
      </Headline>
    </ContentComponent>
  );
};

This looks very familiar from the way we declare the rendering of a content component in Neos via Fusion and AFX. As it turns out this was already heavily inspired by React and so the code looks very similar.

🤩 Demo time

Let's have a look if we can see a difference:

• As we can see there is no frontend if we open Neos directly.

• When we go to the Next.js server, we can see a rendered frontend.

• When we navigate to other pages there are some nice transitions, which are quite natural to implement, since the outer app component in Next.js persists between the page loads and can be animated.

  Further page loads also are quite instant with a statically generated site - even if not running locally, since Next.js performs some nice tricks like loading all the pre-fetched data when hovering over a link.

Once we log into the Neos backend, we get the preview page rendered by Next.js.

You can perform all the normal editing tasks:

• We can update content
• We can insert new elements
• We can update inspector properties
• Updating images also works

Now let's update the title of a document and publish:

• It's changed on the actual page
• But if we go to other pages, the menu item isn't updated
• Let's have a look at regeneration of statically generated pages and how that actually works

How can we invalidate cached content? And when?

What do we need to invalidate? And when?

• If we use static site generation in Next.js, basically a static version of a page is created - that means no data fetching is needed for further requests and no access to the Neos content API is performed.
• This means however, that when something in the underlying data changed, the page needs to be regenerated.
• Next.js offers incremental static regeneration (ISR) for this case, where an API route can be used to revalidate paths. These will be re-generated by calling getStaticProps from the content API and rendering the page again.
• Only Neos knows when content changed, so this API route needs to be called by some mechanism in Neos.

This is what needs to happen when changed nodes are published in Neos:

1. Neos calls the API route in Next.js, which is secured by a token, and sends the route paths of the closest document nodes for each published node.
2. Next.js will generate each of these paths again and fetch details from the content API.

This works if a change to a node only affects the outcome of the page for that document node.

But what if the page output depends on nodes outside of that document node?

• This happens frequently, for example with menu items.

• We discussed a lot of different approaches, including some kind of tagging like Neos does in the content cache to capture dependencies.

  The biggest drawback of such a solution would be, that these tags would need to be defined in Neos and we need to store some state to find all paths that used data with one of these tags. Which would add a lot of complexity.

Why can't we regenerate everything for each change?

• It could take long if we have many nodes.
• There might be a long delay until the actual change is visible.
• There is no built-in queue mechanism in Next.js to perform background tasks.

Turns out, these are merely technical challenges, and all we needed was a background service that implements a smart priority queue:

It's called Grazer and implements these special properties:

• Paths are unique.
• Explicit updates have more priority than regenerating the rest of the pages for consistency.
• And previous updates have more priority than new updates.

It's easy to deploy beside a Next.js instance and acts as a proxy to the revalidate API. It also allows to shift the static generation from build time to after the deployment - and pre-generates all pages in the background.

With these things in place we can now deliver the full editing experience with everything working as you would expect - even for larger and more complex sites.

• Without any specific cache configuration
• And rendering fully defined in Next.js, without any Fusion for HTML in Neos.

Packaging it up

One final step was to package up these solutions in packages, that capture the boilerplate and ideas, but allow for a flexible use.

• Zebra is the NPM package that provides the data fetching and rendering code for a Next.js project.
• Networkteam.Neos.Next is a Composer package that provides a Next.js based preview renderer for nodes - for out of band rendering - and revalidate notifier for invalidation.
• Networkteam.Neos.ContentApi is a Composer package that implements the Fusion based content API.
• And finally Grazer which is optional, but an important piece to get the revalidation of content right.

Try it for yourself!

For getting started with these packages, we provide a demo repository, which bundles a Next.js and Neos installation with a working setup:

github.com/networkteam/zebra-demo

To wrap it all up:

• With the content API and Zebra, you can use Next.js for the rendering of a Neos site without using Fusion to generate HTML.
• By using the components and hooks from the Zebra package you can have the full editing experience while gaining the architectural advantage of an headless approach.
• And lastly, you will have a modern and streamlined process for your project with a great developer experience.

We hope you give it a try and our team is happy about your feedback or helping you with your first Next.js and Neos project.

Thank you!

Special thanks also to Philip Schmidt for initiating and pushing the full-editing approach and implementing the first Zebra project in his spare-time.