neatComponents Enterprise Accelerator architecture
Large and Very Large Sites: providing enhanced scalability

Introduction

neatComponents generally builds pages using database queries. Whilst these are usually cached internally, the sheer volume of new queries needing to be executed can mean that the page creation time is non-trivial, which slows the response to users when the server is under load.

The Enterprise Accelerator architecture uses separate machines acting as intelligent caches to reduce the load on the base nc server.

Generic server-side caching software solutions already exist in the market, however these cannot help where pages are personalised and behind security, which is often the case with nc sites.

How it works

The servers involved are split into nc servers and cache machines.

For simplicity let us first assume there is just one site, hosted on one nc server.

Without the EA system, web requests for resources would go directly to it. However with the EA system, the DNS is set so that client requests go to the cache machine(s).

The cache machine checks whether it has the required resource, and if it does, responds with it.

If it does not, it requests missing resources, before caching them, and responds with them.

Scaling up – adding more cache machines

If there is more than one cache machine the cache machine first asks the other cache machines (its peers) if they have the resource, as it is cheaper to fetch the resource from another cache than forcing the nc server to regenerate it. Only if no cache responds with the resource does the nc server have to be asked for it.

If a cache machine receives a resource request from a peer, and it does not have the resource, it does not attempt to obtain the resource itself, but responds that it does not have it. (Only if a request comes directly from a user does it ever request it from the nc server – thus preventing the nc server receiving multiple requests from different caches as a result of a single client request)

Logging

The cache machines are responsible for logging resource requests, and passing this data back to the nc server. This is passed back in batches to be as efficient as possible, and gives the nc server the same level of detail as if the requests arrived directly. (This is another advantage over generic caches, which hide the traffic from the nc server.)

Use of DNS

DNS A-Records are used to list the cache machines, so that they attract the requests from client browsers in a simply load-balanced manner. The cache machines also inspect the DNS to be aware of their peers for requesting missing resources.

The cache machines also need to know where the nc server is. This can be configured directly in a separate DNS record (eg a TXT record)

It is important to keep the DNS TTL as low as practical to enable changes to be noticed quickly when new cache machines are added (or removed) to handle changes in load volume.

DNS load balancing can be simple round-robin, where clients are randomly allocated to cache machines, or a more deterministic approach can be used. The latter will increase the likelihood of a cache containing the required data if it is personalised to a particular user.

Deterministic DNS approaches based on geographic location will also reduce latency between the client and the cache, providing the essence of a CDN (Content Delivery Network). This will be particularly effective for resources with long TTLs, for example images.

Scaling up

There is no limit to the number of cache machines which can work against a nc machine, although clearly the nc machine will eventually be saturated by requests for uncached resources from the caches.

'Zero' configuration

Theoretically, a cache machine needs no configuring at all. If it receives a request, it can obtain all the information it needs from the public DNS records. However a layer of security is needed to prevent a cache machine being used without authorisation. This authentication can be common to all the cache machines within a network, so a configured cache machine can be imaged and replicated for rapid scaling.

Flexibility

Each cache machine can handle multiple sites, (which can be based on different nc servers). Since there are no configuration changes needed to be made to the servers this is simply a matter of modifying the DNS to include them.

Operating system

The cache machines are based on BSD to enable zero-cost upstream licensing and ease of deployment.

An ISO image of the cache machine can be downloaded from the nc server, with the organisation key pre-configured within it.

Authentication

As noted above, it is necessary to prevent caches being used without permission. To lock them to only work with the correct ns servers, and not be 'leeched' for use on others, each organisation has an organisation key, which is used to sign each request made by the cache to the other machines (both caches and nc servers) operated by the same organisation.

The benefit of using the same key for all machines operated by the same organisation (rather than having separate ones for each nc server) is that a cache machine can be reallocated from use with one nc server to another, or be shared for use with several nc servers, by simply changing the DNS records to route new requests from end users, thus preserving the 'zero-configuration' principle.

CDN edition

Content Delivery Network operators who wish to provide nc cache machines for use by multiple independent nc server owners can obtain a special edition of the cache.

This allows the usage of the cache to be controlled by a separate DNS-based method, preserving the zero administration principle to allow rapid deployment, but placing the control over which domain names can be cached with the CDN operator.

Please contact us for details and pricing