Zip::InputStream can not read content with extended local header (0x08074b50)

[artembaikuzin]

Inputs

I have this weird zip archive which has 0x08074b50 header in the beginning of file:

invalid.zip

hexdump invalid.zip
0000000 50 4b 07 08 50 4b 03 04 14 00 00 00 08 00 35 8e
0000010 87 4b d5 3c fa dc 0a 00 00 00 0f 00 00 00 08 00

Also, I have usual zip archive with exact same content, but with the normal header:

valid.zip

hexdump valid.zip
0000000 50 4b 03 04 14 00 00 00 08 00 35 8e 87 4b d5 3c
0000010 fa dc 0a 00 00 00 0f 00 00 00 08 00 1c 00 74 65

Issue

Zip::InputStream can't read invalid.zip file and silently returns nil, but Zip::File.open works just fine.

Source code

require 'zip'

Zip::InputStream.open(File.open('invalid.zip')) do |zip_io|
  while (entry = zip_io.get_next_entry)
    puts "InputStream, invalid.zip: #{zip_io.read}"
  end
end

Zip::InputStream.open(File.open('valid.zip')) do |zip_io|
  while (entry = zip_io.get_next_entry)
    puts "InputStream, valid.zip: #{zip_io.read}"
  end
end

Zip::File.open('invalid.zip') do |zip_file|
  zip_file.each do |entry|
    puts "File.open, invalid.zip: #{entry.get_input_stream.read}"
  end
end

Zip::File.open('valid.zip') do |zip_file|
  zip_file.each do |entry|
    puts "File.open, valid.zip: #{entry.get_input_stream.read}"
  end
end

Output:

InputStream, valid.zip: test test test
File.open, invalid.zip: test test test
File.open, valid.zip: test test test

[artembaikuzin]

Method Zip::File.open_buffer can be used to solve this problem. It reads invalid.zip archive from buffer without an issue:

File.open 'invalid.zip', 'rb' do |io|
  Zip::File.open_buffer(io.read) do |zip_file|
    zip_file.each do |entry|
      puts "File.open_buffer, invalid.zip: #{entry.get_input_stream.read}"
    end
  end
end

File.open_buffer, invalid.zip: test test test

[hainesr]

Hmm. 0x08074b50 at the very start of a file indicates that it is the first in a series of files that makes up a split archive. For it to be a valid split it should have a name suffix of .z01 and there should be at least one other file with the .zip suffix that holds the Central Directory. So this is an invalid file for a number of reasons: it has the Central Directory in it, and it's less than 64K (which is the minimum split size).

Zip::File.open and Zip::File.open_buffer can read it because the Central Directory is in there and they use that to find the local entry data. Zip::InputStream relies on finding the local header signature at the start of the file/stream so can't read it.

Do you think the Zip::InputStream methods should complain/warn rather than fail silently?

[artembaikuzin]

Thank you for clarification! It would be great if Zip::InputStream give some kind of an error / exception in this case.

[hainesr]

I suspect that Zip::InputStream should actually refuse to work with split archives at all because splitting makes no sense in a stream context - it's a file thing really. What does it mean to split a stream? And how do you splice together streams?

I'll think on it, but I suspect Zip::InputStream raising an error the moment it sees 0x08074b50 at the start is the correct behaviour.

[hainesr]

@ybinzu may I use the invalid zip file you posted above as a fixture to test against?

[artembaikuzin]

@ybinzu may I use the invalid zip file you posted above as a fixture to test against?

Sure! No problem.