sending next packet before the last packet is sent

sending next packet before the last packet is sent

Post by yawnmot » Wed, 16 Sep 2009 11:34:23


I'm trying to upload a 1mb file using SFTP and the upload is taking
quite a bit longer than the download. I was thinking that, to speed
it up, I might use non-blocking sockets. Before receiving
confirmation that a portion of a file has been written, I send the
next portion. The thing I'm worried about is this: might the SFTP
server silently drop packets?

The fact that some packets might wind up being sent out-of-order isn't
really an issue - each SFTP packets contains the starting address and
the length.

Anyway, any ideas?
 
 
 

sending next packet before the last packet is sent

Post by Barry Marg » Wed, 16 Sep 2009 14:02:48

In article
< XXXX@XXXXX.COM >,



You seem to be assuming that TCP blocks if you haven't received an
acknowledgement. This is not true. It only blocks if the lock socket
buffer is full, because you're sending much faster than the receiver can
read.

Changing to non-blocking sockets doesn't affect this. The difference is
that when you would have blocked, write() will instead return an
EWOULDBLOCK error, and then you'll have to use select() to wait for
buffer space to free up. Either way, you can't send faster than the
server can receive.


SFTP runs over TCP, which ensures that everything is received in the
order it was sent, with no gaps.

--
Barry Margolin, XXXX@XXXXX.COM
Arlington, MA
*** PLEASE don't copy me on replies, I'll read them in the group ***

 
 
 

sending next packet before the last packet is sent

Post by David Schw » Thu, 17 Sep 2009 13:43:43


You seem to have quite a few misconceptions, but the one that really
stands out is that you seem to think that there's a one-to-one
correspondence between SFTP packets and packets on the wire. Nothing
"glues" the bytes of an SFTP packet together below the SFTP layer. The
TCP layer can segment them as it wishes and the IP layer can fragment
those segments into packets however it wishes.

SFTP "packets" can never arrive out of order because they exist in an
ordered byte stream.

Also, you seem to have a confusion about blocking versus non-blocking
sockets. A blocking socket will only block if it's not possible to
make forward progress. If it's not possible to make forward progress,
a non-blocking socket won't block, but it won't make forward progress
either (since that isn't possible). The only reason to use non-
blocking sockets is if you don't want to block when forward progress
isn't possible (usually because there's something *else* you might
want to do). If there's nothing else you might want to do, there's no
reason to use non-blocking sockets.

As for whether the SFTP server might drop packets, it's not clear what
you're talking about. If you're talking about SFTP protocol data
units, how could it drop them? What mechanism do you imagine might
cause that? If you're talking about network packets, it might, but
they'll get retransmitted anyway, and it's very unlikely that anything
you do on the sending side will affect this (because that's handled by
TCP and it doesn't give you that level of control anyway).

DS
 
 
 

sending next packet before the last packet is sent

Post by Tom Einert » Sun, 20 Sep 2009 02:24:29


XXXX@XXXXX.COM ...

...

Both Barry and David pointed out good reasons why using non-blocking
sockets won't improve your performance. Baring a bug in the TCP or
SFTP implemtation, SFTP should be able to transfer data at close to the
theoretical maximum that your network (and computer) can sustain.

So rather than starting by changing some code, I would start by
treating this as a performance problem and analyze why the upload
speeds are so slow. A network trace will usually show where the
bottleneck is. If you don't know how to interpret such a trace, I'm
sure there is someone on this group who could help interpret the
output.

--
Tom Einertson E-mail: XXXX@XXXXX.COM
SIEMENS Power Transmission & Distribution Phone: (952) 607-2244
Energy Management & Automation Division Fax: (952) 607-2018
10900 Wayzata Boulevard, Suite 400
Minnetonka, MN, 55305
 
 
 

sending next packet before the last packet is sent

Post by kavi » Mon, 21 Sep 2009 15:32:31


< If not related to sftp>
Are you using some custom sftp ?
If not, There may be some issues regarding where you are trying to
write these 1 MB file,

If it is a flash, you may need to check how fast you are able to
write in the flash ?
just try to copy a local file from one directory to other and see how
long it takes?

Another idea is to check the network latency in general. If it is
slow, you may need to address it first.

Kavi
 
 
 

sending next packet before the last packet is sent

Post by Jorgen Gra » Tue, 22 Sep 2009 16:06:47


By the way, if I recall correctly SFTP is a bit weakly defined, and
the name being used to cover various "looks like my old FTP client, but
works over ssh" scenarios, not just

T. Ylonen and S. Lehtinen, SSH File Transfer Protocol,
draft-ietf-secsh-filexfer-00.txt, January 2001, work in progress
material.

Maybe the OpenSsh implementation is the defacto standard.


Good points. Other possible reasons for the speed difference:

- the OP does indeed have assymetric network speed. "Upload is taking
quite a bit longer than the download" describes my ADSL connection
pretty well.

- slow CPU on the downlink side. SSH spends a lot of time encrypting
and compressing data, and if you have reasonably fast networking,
CPU can easily become the bottleneck. I guess the sender needs more
CPU than the receiver, so a slow host A might be the bottleneck
A->B, but not B->A.

/Jorgen

--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
 
 
 

sending next packet before the last packet is sent

Post by glen herrm » Fri, 09 Oct 2009 03:21:28


< I'm trying to upload a 1mb file using SFTP and the upload is taking
< quite a bit longer than the download. I was thinking that, to speed
< it up, I might use non-blocking sockets. Before receiving
< confirmation that a portion of a file has been written, I send the
< next portion. The thing I'm worried about is this: might the SFTP
< server silently drop packets?

Many now use connections with asymmetric bandwidth, including the
usual cable and DSL systems.

The cable TV system was designed to bring many wide band (6MHz)
signals to homes, but allows for a small bandwidth upstream.
As the signal has to go through amplifiers, it must be possible
to separate the two directions at each amplifier point. The
lowest broadcast TV channel starts at 54MHz, so the split point
was chosen somewhat below that.

-- glen