Revisiting FTP Basics

Table of contents

1. Fundamentals

2. How active FTP works

2.1 Flow chart of active FTP

2.2 Flow chart explanation

2.3 Network trace

3. How passive FTP works

3.1 Flow chart of passive FTP

3.2 Flow chart explanation

3.3 Network trace

4. Firewall

4.1 Firewall setting for active FTP

              4.1.1 Server side firewall rule

4.2 Firewall setting for passive FTP

             4.2.1 Server side firewall setting

5. Implicit and explicit FTPS

5.1 How explicit FTPS works

                          5.1.1 Output of explicit FTPS in FileZilla

                          5.1.2 Output of FTPS logs after running Log Parser tool

5.2 How implicit FTPS works

                         5.2.1 Output of implicit FTPS in FileZilla

                          5.2.2 Output of FTPS logs after running Log Parser tool

6. Tools used to troubleshoot FTP issues

6.1 NetMon trace

6.2 ETW trace

6.3 DebugView

6.4 Port Query

FTP Overview

1. Fundamentals

FTP stands for File Transfer Protocol. In short, FTP is a protocol for transferring files over the Internet, which uses the TCP/IP protocols to enable the data transfer. There is no UDP component involved in FTP. The FTP client initiates a connection to a remote computer running the FTP “server”.

Unlike HTTP and other protocols used on the Internet, the FTP protocol uses a minimum of two connections during a session: a 'data' port and a 'command' port (also known as the control port). By default, TCP port 21 is used on the server for the control connection, but the data connection is determined by the method that the client uses to connect to the server

FTP operates in two modes: active and passive. These two modes differ in the way the client / server chooses the port for data (file) transfer.

2. How Active FTP works

In active FTP, the client machine connects to the FTP server (control port 21) from a random port (port>1023). The client also sends its IP:Port to the server on which it wants to listen. The server then calculates the port, and then transfers the data from (data port 20) to the (port>1023) port calculated from IP:Port combination. The client then acknowledges this transfer.

2.1 Flow chart of Active FTP

2.1.1 Flow chart explanation

1. The client sends a request from a random port, along with the IP:Port combination to the control port (21) of the server. In the figure above, the IP is the IP address of the client, and port is the client port that the server should connect to and through which it wants to listen.

2. The server acknowledges the client’s request.

3. The server initiates a brand-new session with the port (calculated from IP:Port combination in step 1) from its data port ( 20 ).

4. Then the client acknowledges this data received from the server.

The following is a typical sequence for an active-mode FTP connection:

2.2 Network trace

Below is the client side trace in FTP active mode. Apply the filter “ftp”, and then follow the TCP conversation.

The first trace indicates the TCP level handshake between the client and server .

Below, the client is sending the request from port 3471 to the server command port 21 and requesting for a password. Here the password is sent in clear text.

After authenticating the client, the client sends the IP:Port combination to the server. The port which the client likes to listen on will be calculated using the 5^th octet multiplied by 256 and adding the 6^th octet: (13*256)+145 =3473.

Again, the TCP hand shake happens with the port calculated from the IP:Port combination (3473).

After acknowledgement, the data transfer happens from the server data port 20 to the client port.

3. How Passive FTP works

In passive FTP, the client machine connects to the (control port 21) FTP server from a random port (port>1023). The server sends its acknowledgement to the client, and it sends the IP:Port combination. The client then communicates with the server over this port, and data transfer happens over this port. The server then sends the acknowledgement to the client.

3.1 Flow chart of Passive FTP

3.1.1 Flow chart explanation

1. The client sends a PASV command from a random port to the control port of the server (21).

2. The server acknowledges this and sends the IP:Port combination to the client.

3. The client then initiates a brand-new session to the port (calculated from IP:Port combination in step 2) from a random port.

4. Then, the server sends the acknowledgement to the client.

The following is a typical sequence for a passive-mode FTP connection:  

3.2 Output of the network trace

The first trace below indicates the TCP level handshake between the client and server . The client sends the PASV command to whichever server responds with the IP:Port combination: (18*256+232 =4840)

Then the data transfer happens with this port (4840) and the client port.

4. Firewall

A firewall is software or hardware that checks information coming from the Internet or a network, and then either blocks it, or allows it to pass through to your computer, depending on the rules defined in the firewall settings.

FTP commands are sent over a particular channel called control channel. That is the one that typically connects to the well-known FTP port 21. Any data transfer, such as directory listing, upload and download, happens on a secondary channel called data channel.
To open port 21 on a firewall is an easy task. But having port 21 opened only means that clients will be able to connect to FTP server, authenticate successfully, create, and delete directories. This does not meant that clients will be able to see directory listings, or be able to upload/download files. This is because data connections for the FTP server are not allowed to pass through the firewall.

Many firewalls simplify this challenge with data connections by scanning FTP traffic and dynamically allowing data connections through. Some firewalls enable such filters by default, but this is not always the case. These firewall filters are able to detect what ports are going to be used for data transfers. and temporarily opens them on the firewall so that clients can open data connections.

Many firewalls do not accept new connections through an external interface. The firewall detects these connections as unsolicited connection attempts and drops them. Standard mode FTP clients do not work in this environment because the FTP server must make a new connection request to the FTP client.
Firewall administrators may not want to use passive mode FTP servers because the FTP server can open any ephemeral port number. Firewall configurations that allow full access to all ephemeral ports for unsolicited connections may be considered unsecured.

Reference: https://blogs.iis.net/jaroslad/archive/2007/09/29/windows-firewall-setup-for-microsoft-ftp-publishing-service-for-iis-7-0.aspx

4.1 Firewall setting for Active FTP.

Information for client side and server side firewalls are outlined below.

4.1.1 Server side firewall rule:

In the server side firewall rule, we need to make sure that control port 21 and data port 20 are opened in the inbound and outbound rule section respectively.

4.2 Firewall setting for Passive FTP

Information for the client side and server side firewall settings are outlined below.

4.2.1 Server side firewall setting

With the server side setting, the server makes a decision of sending its port to the client, so we need to make sure that only that range of port is available in the inbound and outbound rule.

For example, if you want the server to choose a port between the 5000-10000 ranges, make sure this range is allowed in both inbound and outbound rules. Also, specify this range under the FTP Firewall Support section at the server level in IIS Manager.

In the figure below, we made all the ports between 1024 and 65535 ports available.

5. Implicit and Explicit FTPS

One of the many limitations of FTPs is a general lack of security. For example, user names and passwords are transmitted in clear text, data is transferred with no encryption, etc. In order to address this situation, FTP over SSL (FTPS) was introduced.

FTPS (SSL/TLS) has two types:

          1. Implicit

          2. Explicit

In explicit FTPS, the client connects to the control FTP port (port 21) and explicitly switches into secure (SSL/TLS) mode with "AUTH TLS".

On the other hand, in implicit FTPS, SSL/TLS mode is assumed right from the start of the connection, and normally listens on TCP port 990, rather than 21.

FTPS adds encryption security to FTP connections. Therefore, the benefit is that you can use FTP technology with SSL encryption to transfer data.

Reference: https://blogs.iis.net/robert_mcmurray/archive/2008/11/10/ftp-clients-part-2-explicit-ftps-versus-implicit-ftps.aspx

5.1 How Explicit FTPS works

The FTP client connects over the control/command channel (port 21), and then the client can negotiate SSL for either the command/control channel, or the data channel using new FTP commands like AUTH, PROT, CCC, etc.

Reference: https://blogs.msdn.com/b/robert_mcmurray/archive/2008/11/10/ftp-clients-part-2-explicit-ftps-versus-implicit-ftps.aspx

5.1.1 Output of explicit FTPS in FileZilla:

5.1.2 Output of FTPS logs after running Log Parser tool:

5.2 How implicit FTP works

Implicit FTPS takes SSL one step further than simply requiring that SSL-related commands be sent first, like you can with Explicit SSL. With implicit FTPS, an SSL handshake must be negotiated before any FTP commands can be sent by the client. In addition, even though explicit FTPS allows the client to arbitrarily decide whether to use SSL, implicit FTPS requires that the entire FTP session be encrypted. Basically, the way that implicit FTPS works is that an FTP client connects to the command/control channel, in this case using port 990, and immediately performs an SSL handshake. After SSL has been negotiated, additional FTP commands for the session can be sent by the FTP client.

Reference:https://blogs.iis.net/robert_mcmurray/archive/2008/11/10/ftp-clients-part-2-explicit-ftps-versus-implicit-ftps.aspx

5.2.1 Output of implicit FTP in FileZilla:

5.2.2 Output of FTPS logs after running Log Parser tool:

6. Tools used to troubleshoot FTP issues

Information on some of the tools used to troubleshoot FTP issues are outlined below.

6.1 NetMon trace

This tool helps understand the ports that are being used, and also finds out if there is any firewall or and networking device which is intercepting the client-server request. This tool will not help for the FTP over SSL because the packets are encrypted.

6.2 ETW Trace

You use this tool in order to enable FTP ETW trace. To do that, open Command Prompt in the server and follow the steps below (for more details, visit Collecting ETW trace for FTP).

1. Type “logman start "ftp" -p "IIS: Ftp Server" 255 5 –ets”.

2. Reproduce the issue.

3. To stop the trace, type “logman stop "ftp" –ets”

4. Once you have the FTP ETW trace, use LogParser to parse the trace file. The command is:

LogParser.exe "select EventTypeName, UserData from ftp.etl" -e 2 -o:DATAGRID -compactModeSep " | " -rtp 20

6.3 DebugView

DebugView is an application that lets you monitor the debug output on your local system, or any computer on the network that you can reach via TCP/IP. It is capable of displaying both kernel-mode and Win32 debug output, so you don't need a debugger to catch the debug output your applications or device drivers generate. You also do not need to modify your applications or drivers to use non-standard debug output APIs (for more details, visit DebugView ).

6.4 Port Query

This tool, also known as PortQry, is a command-line utility that you can use to help troubleshoot TCP/IP connectivity issues. This utility reports the port status of target TCP and User Datagram Protocol (UDP) ports on a local computer or on a remote computer. It checks if the ports are blocked or not. This tool has a user Interface which can downloaded here.

Comments

• Anonymous
  September 05, 2014
  Great article, thanks for the refresher and concise explanations:)

• Anonymous
  September 05, 2014
  Full information at one place...very useful one Naveen

• Anonymous
  September 01, 2015
  Great article to begin with FTP !!!

• Anonymous
  December 23, 2015
  Thanks Naveen for this article. Easy and detailed information.