Virtually Infinite Scrolling with Angular

In this article we will take a look at some common patterns to improve performance when implementing an infinite scroller with Angular.

Published 1 year ago
13 min read
2,729 views

This article has been archived as it contains outdated information, non-working examples, or because I am no longer happy with the content. It has been made unlisted, and can still be accessed with a direct link.

Overview

Infinite scrollers have been around for a really long time. The concept is simple, as you scroll down the page additional content is fetched and added to the bottom, essentially making it so the scrolling never ends. Implementing an infinite scroller is quite straightforward, but without careful consideration, you might end up negatively affecting page performance. By the time you’ve loaded the page and scrolled through enough to re-fetch content, chances are you have a lot of elements added that are no longer needed which is amplified by the number of times you reach the bottom. By the time you know it you could have hundreds of appended elements with the user only seeing a handful of them at a time, which can result in a poor experience.

So why would you want to implement one of these? You may just have a lot of content, or you might want to build an application that is similar to Instagram or TikTok that encourages users to scroll a lot. Whichever your reason in this article we'll take a look at some common patterns to improve this experience so you can scroll as much as you want without running into performance problems. We'll also go over some code samples, and use Angular and RxJs to implement this pattern in a web application. You can get an idea of what we'll be building here, or if you want to just see the code click here.

DOM Elements In Inspect Element
Ideally we want to avoid this.

One of the most optimized ways around this problem is to implement something called a virtual scroller. What is a virtual scroller you ask? It’s the concept of faking the scroll depth as you scroll, while swapping out the contents of a handful of elements behind the scenes. Let’s say you have five elements full of content, where only three of them are showing within the viewport at a single time. Once element 2 through 4 is visible, the element in the 1st position is ready to be recycled, as a result it’s content is swapped out and it’s moved below the element in the 5th position. As you scroll the depth is maintained by adding some height to the container to ensure that the viewport doesn’t bounce once the element swap occurs. In addition logic behind the scenes is keeping track of the entire list of content, associating specific scroll depths with an index to maintain the ordering as you scroll up and down on the page. The reason the scroll is "virtual" is because the logic is providing the calculated page depth as opposed to the elements, the element re-ordering and height changes make it feel that way when in reality an orchestra or replacements are happening to limit the amount of appended nodes.

The recycled nodes in the following image have their content replaced once they are outside of the viewport area with the content from the unrendered ones. There's a degree of buffering so you never see the swap occur while it's in the viewport.

Diagram showing virtual scrolling
The areas within the recycled nodes section are all that is renderd to the DOM. As the viewport shifts, the items that are not rendered get pushed into the recycled nodes area to give the illusion of scrolling.

The Vanilla Way

Before we dive into a framework-oriented example, let’s quickly look at how this would work with vanilla JavaScript. The following class has the job of determining which items within a given range of indexes are in the viewport at a given time. It does this by dividing the scroll containers' scroll depth and height against each row height after a scroll event occurs to get startIndex and endIndex, which are then passed to the callback function.

class VirtualScroller {
  constructor(options) {
    this.options = options
    this.scrollTop = 0
    this.itemHeight = 0
    this.itemCount = 0
    this.startIndex = 0
    this.endIndex = 0
    this.init()
  }

  /**
   * Once the class is instantiated, bind scroll event handlers.
   */
  init() {
    const { scrollContainer, contentContainer, itemHeight, itemCount } =
      this.options

    this.scrollContainer = scrollContainer
    this.contentContainer = contentContainer
    this.itemHeight = itemHeight
    this.itemCount = itemCount

    this.update()

    this.scrollContainer.addEventListener(
      'scroll',
      this.handleScroll.bind(this),
    )
  }

  /**
   * Gets the current top value after each scroll.
   */
  update() {
    this.scrollTop = this.scrollContainer.scrollTop
  }

  /**
   * On every scroll event update the scroll top value and
   * determine which items to render.
   */
  handleScroll() {
    this.update()
    this.render()
  }

  /**
   * Handles the rendering of the items.
   */
  render() {
    const { onRender } = this.options

    this.startIndex = Math.floor(this.scrollTop / this.itemHeight)
    this.endIndex = Math.min(
      this.itemCount - 1,
      Math.ceil(
        (this.scrollTop + this.scrollContainer.offsetHeight) / this.itemHeight,
      ),
    )

    onRender(this.startIndex, this.endIndex)
  }
}

When we instantiate the class there are a few things we'll need to provide upfront, such as how many items we want to render and the pre-determined height of the rows. We'll also define our callback method which appends the items to the scroll container. It does this using a for loop that starts at startIndex and ends at endIndex, using the index of each iteration to pick an item from our array. This is a fairly basic example, and if you were implementing this for real you'd probably want to offload a lot of this logic into the class.

const scrollContainer = document.querySelector('.scroll-container')
const contentContainer = document.querySelector('.content-container')

// Define your items here!
const items = []
const itemCount = items.length
const itemHeight = 50

const virtualScroller = new VirtualScroller({
  scrollContainer,
  contentContainer,
  itemHeight,
  itemCount,
  onRender(startIndex, endIndex) {
    contentContainer.innerHTML = ''

    for (let i = startIndex; i <= endIndex; i++) {
      const itemElement = document.createElement('div')
      itemElement.classList.add('item')
      itemElement.style.top = `${i * itemHeight}px`
      itemElement.innerText = items[i]
      contentContainer.appendChild(itemElement)
    }
  },
})

As a result, we get a fairly basic virtual scroller. You can see as we scroll there's only a handful of elements rendered at a single time. In reality, we're plucking from an array of almost 1,000 randomly generated items.

weeeee!

It's worth noting that this example does not calculate row heights dynamically. You can write a virtual scroller that does, but if the intention is to leverage this as part of an infinite scroller as well you need to be careful that you don't re-calculate every row each time something new is fetched, but instead, just the rows that newly added, otherwise you may run into some significant performance problems.

The Angular RxJs Way

I recently set out to build an application that integrates an infinite virtual scroller. My goal was to build something that could fetch something from Reddit, display a set of videos and images, and make it so I could mindlessly scroll. The application needed three primary components, a way to type in a subreddit name, tab controls for the different filter types, and an infinite scrolling mechanism. By typing in a new page or changing the filter the content should reset, and the infinite scrolling should begin again.

Since originally writing this Reddit has made steps to paywall their API. As a result you may have mixed results with the following code examples.

But why Angular? I’ve been deep diving into Angular and RxJs over the last few years and I think it does a particularly good job at implementing patterns of this type. Not only are there fully supported utility libraries, but RxJs streams allow you to offload a lot of the heavy lifting away from your templates for a clear separation of concerns. I also use it pretty much daily these days, so you know... the added benefit of familiarity was there too.

Let’s start by building a service that can fetch data from the API. To get the data we’ll use the HttpClientModule available as part of Angular, and put a utility method in a service to return an observable with the requested data. Before the data is returned it gets piped through a couple of RxJs operators to ensure that the structure we're getting is flat enough that we only need to iterate a single time in our template. We’ll need this method to accept a few properties that can be passed to the request.

private getSubRedditContent({
  name,
  filter,
  page = '',
}: IRedditRequestOptions): Observable<IRedditResult[]> {
  const path = new URL(`https://www.reddit.com/${name}/${filter}/.json`)
  path.searchParams.append(
    RedditRequestParameters.LIMIT,
    RedditService.MAX_CONTENT_FETCH.toString()
  )

  // If page is provided it gets appended to the query to ensure we're not re-fetching the same content.
  if (page) {
    path.searchParams.append('after', page)
  }

  return this.http.get<IRedditResultNatural>(path.toString()).pipe(
    map(result =>
      // Flatten!
      result.data.children
        .map(item => item.data)
    )
  )
}

In that same service, we’ll set up a couple of other observables that will hold the state for the requested name, filter and page properties.

private readonly _subRedditName$ = new BehaviorSubject(
  RedditService.DEFAULT_SUBREDDIT
)
private readonly _subRedditPage$ = new BehaviorSubject(
  RedditService.DEFAULT_PAGE
)
private readonly _subRedditFilter$ = new BehaviorSubject(RedditFilter.HOT)


public getSubRedditName(): Observable<string> {
  return this._subRedditName$.asObservable()
}

public setSubRedditName(name: string): void {
  this._subRedditName$.next(name)
}

public getSubRedditFilter(): Observable<RedditFilter> {
  return this._subRedditFilter$.asObservable()
}

public setSubRedditFilter(filter: RedditFilter): void {
  this._subRedditFilter$.next(filter)
}

public getSubRedditPage(): Observable<string> {
  return this._subRedditPage$.asObservable()
}

public setSubRedditPage(page: string): void {
  this._subRedditPage$.next(page)
}

These utilities will be used to transmit the requested content. For example the search bar can call setSubRedditName to emit a new value to the subscribers of the _subRedditName$ observable.

Next up is the primary mechanism of the entire app, the _query$ observable. Using RxJs you can combine these streams into a single one that can simply be subscribed to within the template using the async pipe. The following might look a bit confusing at first, but essentially it checks for any changes to either _subRedditName$, _subRedditPage$ and _subRedditFilter$, and fires off a stream that re-fetches content using the getSubRedditContent utility.

private readonly _query$: Observable<IRedditQuery>

public constructor(private readonly http: HttpClient) {
  this._query$ = combineLatest([
    this.getSubRedditName(),
    this.getSubRedditFilter(),
  ]).pipe(
    switchMap(([name, filter, subFilter]) =>
      this.getSubRedditPage().pipe(
        mergeMap(page =>
          this.getSubRedditContent({
            name,
            filter,
            page
          })
        ),

        /**
          * If the page observable emits instead of creating a new list it
          * will instead concat the previous results together with the new ones.
          * This is used for infinite scrolling pagination.
          */
        scan(
          (acc, curr) => ({
            results: curr.length ? acc.results.concat(curr) : acc.results,
            // Sets the next page so it can be referenced again to fetch the next page.
            nextPage: curr[curr.length - 1]?.id
          }),
          {
            results: [] as IRedditResult[],
            nextPage: undefined
          } as IRedditQuery
        )
      )
    )
   )
}

public getQuery(): Observable<IRedditQuery> {
  return this._query$.asObservable();
}

There is one exception, however, and that's if _subRedditPage$ changes. If the page changes, the scan operator is used to merge the previous stream results with the current one, essentially creating a large array of data that spans multiple pages. The idea behind this is that you can scroll as much as you like, and only in cases where the filter or the name changes do we completely re-fetch to get fresh content. You might be able to see where I'm going with this, but if you implement this against a virtual scroller you can essentially scroll forever.

In the template, all that needs to be done is create a property that calls RedditService.getQuery() and subscribe to it using the async pipe. All of the mechanisms are housed outside of the component which means if we wanted to use the same data in another part of the site we could do so by subscribing to the same observable!

<div *ngFor="let result of query$ | async"></div>

Back to the Virtual Part

Angular offers a virtual scroller component as part of the @angular/cdk package. To use it simply run npm i @angular/cdk and import the module into your component. This package is maintained by the Angular team, so no need to write any fancy logic here.

import {
  CdkVirtualScrollViewport,
  ScrollingModule,
} from '@angular/cdk/scrolling'

@Component({
  selector: 'app-search-results',
  standalone: true,
  changeDetection: ChangeDetectionStrategy.OnPush,
  imports: [CommonModule, ScrollingModule],
  schemas: [CUSTOM_ELEMENTS_SCHEMA],
  templateUrl: './search-results.component.html',
})
export class SearchResultsComponent {}

We can then wrap the content within the component using the previously set query$ observable. Note the slightly different syntax here for looping, you should instead use *cdkVirtualFor.

<ng-container *ngIf="{query: query$ | async} as data">
  <ul *ngIf="data.query">
    <cdk-virtual-scroll-viewport scrollWindow [itemSize]="itemSize">
      <li *cdkVirtualFor="let result of data.query.results" [height]="itemSize">
        <app-media [content]="result" [size]="itemSize"></app-media>
      </li>
    </cdk-virtual-scroll-viewport>
  </ul>
</ng-container>

There are a bunch of different strategies available with the cdk-virtual-scroller component, but one that you might be interested in particular is the autoheight one. You can install the experimental module by instead installing @angular/cdk-experimental. I’ve given this a try myself and I’ve run into some issues while appending content mid-scroll, bare in mind it's considered experimental for a reason.

What about the infinite bit?

The last part here is to rig up the mechanism to fetch more content when we're near the bottom of the current viewport. To do this you can attach a (scrollIndexChange) handler to cdk-virtual-scroll-viewport. If you look at the previous code samples you may recall seeing where nextPage is set as a property of the query$ stream. This is where it comes into play, and it gets used here to emit to the page$ observable to start the content fetch for the next page of content.

<cdk-virtual-scroll-viewport
  role="presentation"
  scrollWindow
  [itemSize]="itemSize"
  (scrolledIndexChange)="onScroll(data.query.nextPage)"
  className="search-results__viewport"
>
</cdk-virtual-scroll-viewport>

In the component, you can then write a method that sets the next value on the page observable once the depth is past a certain threshold. I found it’s typically best to set this using the less-than-equal operator as it’s possible to skip over indexes if you scroll fast enough.

@ViewChild(CdkVirtualScrollViewport)
public viewPort?: CdkVirtualScrollViewport

public onScroll(nextPage?: string): void {
  if (this.viewPort && nextPage) {
    const end = this.viewPort.getRenderedRange().end
    const total = this.viewPort.getDataLength()

    // If we're close to the bottom, fetch the next page.
    if (end >= total - SearchResultsComponent.FETCH_MINIMUM) {
       this.redditService.setSubRedditPage(nextPage)
    }
  }
}

With all of this rigged up, you'll be left with a virtually infinite scroller!

I can do this all day!

If you want to check out this project you can see the full code on GitHub. You can also test out the app yourself here.