(S)FTP Poller

Secure, Reliable, Managed, File Transfer

Case Studies

The following studies describe how the FTP Poller has been applied to solving problems within a range of situations.

Advertising

This advertising and media company had a large number of remote screens and players that automatically downloaded content from a website and displayed that content on a continuous basis. The problem was that the systems all played content from a single website and the company wanted to display different streams of programs on different screens.

There was also the requirement for putting collections of players into groups. There was also a degree of customisation of programming for each site. For example an advert might be specific to only one shop as part of a local promotion.

A further requirement was to provide a means to support continuous operation so that a player would always find a datasource regardless of any failures at the datacentre.

A simple solution was to have a small backend website for each of the players, a common directory structure across all websites provided a means to define common playout content and further directories were used for local custom content.

A simple windows file server was setup in the main office, this had an identical directory structure to those on the websites. This provided a means for users to drop content into directories on their network, intended for delivery to players via the backend websites.

It was then a simple matter to configure an FTP Poller so that it copied files from the internal master file server out to the small backend website system. This was further enhanced by using multiple backend website systems at 2 further datacentres, which would also be updated by the FTP Poller.

The system produced a considerable saving (estimated at £350,000) on purchasing a content management system and made it possible for staff within the organisation to easily rollout content without the need to understand accessing remote players or dealing with complex logins and passwords. Resilience and scalability was catered for by installing relatively small standalone web servers in a number of low cost hosting sites. This reduced IT costs as there was less dependency on building multiple failover systems within a datacentre with all the doubling up of equipment that would need.

 

Broadcast

This large organisation needed to regularly transfer files originating in house to cooperating entities such as playout centres, subtitling companies or compliance teams. As a worldwide company with many international cable channels, this required dealing with many countries across multiple time zones. The contents vary from contracts and agreements through to extracts from databases such as playlists and legal reporting for regulators. These had to be securely and reliably delivered on time and with a clear audit and diagnostic trail. There were some special cases where playout lists needed to be modified for the correct time zone. There was also need for playout reports to be picked up so that they could be compared against what was intended to be played out. A final requirement was picking up diagnostic reports from failures on broadcast equipment and returning those to the broadcaster.

Our solution used an FTP Poller to completely automate the movement of files between internal departments and external collaborating partners as well as the delivery of reports to broadcasting regulators. The connectivity was implemented using VPN's to give extra security for the transfer of files. Where this was not possible, encrypted files transferred via SFTP gave a similar level of security.

We managed the time zone changes to playout lists by using the pre transfer hook to start a program that modified the original file and dropped copies into a directory for each required time zone.

We arranged for playout reports to be processed after they we retrieved them so that they were electronically compared against the original play list. Any discrepences were then forwarded to the appropriate in house team this reduced their daily work load by an estimated 20%.

With some careful negotiation with the playout centre we were able to retrieve playout diagnostic reports from broadcast equipment and deliver them back to the appropriate department for investigation. This took a burdensome process away from the playout centre and gave the customer direct and immediate visibility of problems. Over a period of three months this allowed the broadcaster to work with the playout centre to reduce transmission problems by 40%.

Total cost savings in the first year were within the region of half a million pounds.

 

Film and TV

Many post production houses work in collaboration with many other companies often sharing workload on joint projects. Our customer had arrangements with over twenty other specialists ranging from video editing and dubbing to special effects and subtitling. Whilst many of the companies involved already had FTP servers, some were not big enough to have such equipment. Some collaborations can last a short period of time such as a few moths, others last for a number of years. Much of the transfer of data was handled using email or if that was impractical (such as HD Video) then manual FTP would be used. This required the setting up of accounts for remote users to pick up files. There was a need for proper reporting and auditing as it was important to be able to show irrefutably who was receiving what files and when transfers had taken place... 

We designed an FTP Poller solution that would allow users to drop files into directories for forwarding to partners who had FTP. We also programmed the solution to regularly look for files to pick up from partners FTP servers. We provided a fully managed solution that covered the FTP Poller and its management as well as supporting FTP accounts for ad-hoc users without FTP servers. In this way remote users without an FTP server could use the companies hub system. We used the post transfer hook to send an email to remote users to remind them that a file was available to them. Overall this simplified operation and from the production houses viewpoint, all users could be worked with in an identical way. This saved time and reduced costs.

 

Mobile Communications

One of the benefits of the way that the FTP Poller works is that the device can be supported as a fully managed service. Layer3 Systems Limited provides a complete one stop solution in design, integration and on going support that reduces the workload on your company minimises total cost of ownership. Our support service uses a start of the art multi data centre support system, allowing us to remotely and securely manage an FTP Poller.
We provide continuous 24 hour support and monitoring for your FTP Poller, often detecting problems on third party networks and resolving issues long before many IT departments are able to focus on a problem. Very often the biggest challenge in handling user driven FTP is that any fault will always appear to be the local system or network. Users are not always able to determine if a fault is related to the Internet or a remote system and so invariably are forced to ask the IT department for help. The FTP Poller and our managed service take these problems away from your IT department, this saves your company time and money.

A large worldwide communications company needed a secure way to transfer network configuration files around the world in support of its wireless network, including holding a full (BC+DR) Business Continuity and Disaster Recovery information on three continents. The infrastructure was already in place and could not be changed. This consisted of a number of SFTP servers around the world. In effect there were a large number of custom scripts, written by different departments, some were highly reliable however they were not easy to manage. Making changes required in depth programming knowledge. Some scripts were very simple and rarely worked without a programmer watching what was happening. There was incomplete auditing and no consistent diagnostic information. A large number of people had a large number of logins and passwords which needed to be centrally managed. This often lead to problems.

We configured an FTP Poller Cluster to use SFTP to securely transfer files all around the world, including dropping copies into secure DR sites in case of problems. This removed the need for high value engineers to waste time trying to move files around and reduced mistakes to near zero.

All scripts were replaced by a centralised managed configuration freeing valuable engineering time. A further benefit of the system was that it could automatically ensure that FTP Servers worldwide, that might have been offline at the time of the latest update, would always have the latest configuration files transferred to them once they were back on line.

Savings were estimated to be in the region of £2.5M in the first year, with ongoing savings in reduction to costs when changes needed to be made to delivery patterns.

 

Travel Industry

A large travel industry company had worked out that distributing Travel Brochures could be made more cost effective if downstream distributors had more warning of what their delivery schedule might look like. With over 100 distributors involved managing  early warning of distribution would not be easy. Although the creation of droplists were available these were only broken down to the major carriers, the far end distributors had to make do with being given delivery notes on the day. The core system generating delivery notes was too expensive to amend and contractually could not be changed.

We provided a solution that involved using FTP Poller to pickup an e copy of all the delivery notes and to process that down into sets of delivery notes per post code. Then produce a summary for each major distributor and for each major distributors sub distribution network.

A Hub systems was put in place to pick up and process the core distribution delivery notes and this then transferred files to satellite systems at the distributors. This in turn then transferred relevant files to the sub distributors.

A key problem was ensuring that the sub distributors all had appropriate systems that could be secure and would handle the delivery of files and provide a simple interface to allow distributors van drivers to print out their summary lists and distribution delivery notes. This was achieved by supplying very small format desktop systems capable of being used in a warehouse environment along with a small printer to create delivery notes.

The FTP Poller provided the core of the system, starting with detecting that the main delivery note file was available and bringing a copy of that file into the system. The post transfer hook was used to fire up the process to split delivery notes by post code. As collections of delivery notes and summaries were dropped into directories named after the distributor who would handle the delivery, the FTP Poller could then pick up those files and transfer them to the systems at the distribution centres and sub distribution warehouses.

The system delivered a 2 day advantage which allowed distributors to manage staffing and logistics more efficiently, savings of the order of £500,000 per year were realised.