#13 open
Jonas Grimfelt

Not supported: Forms without input fields

Reported by Jonas Grimfelt | January 10th, 2009 @ 01:35 PM

In one of my projects I came across that Scrubyt don't support submitting forms without text/select input fields, which is a more common scenario than what one might think.

Example:

Page: zshare.com/download/xxxx


<form action="" method="post" name="form1">
  <input id="referer2" type="hidden" value="" name="referer2"/>
  <input id="download" type="hidden" value="1" name="download"/>
  <input width="219" type="image" height="57" border="0" src="/images/download.gif" name="imageField"/>
</form>

It seems by using instruction such as:


fill_textfield 'user_id', 'foo'

is required to make Scrubyt set a current form, i.e. @@current_form in Scrubyt::Navigation::Mechanize - otherwise it is nil and raises an exception. I consider this as a bug at this point, but quite easy one to solve.

Code that (of course) raise error:


data = Scrubyt::Extractor.define do
  fetch url
  submit

  ...
end

Error:

Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:71:in submit': uninitialized class variable @@current_form in Scrubyt::Navigation::Mechanize (NameError)

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:58:in `submit'
from zdown.rb:79:in `scrape_download_url'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `new'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `define'
...

I tried to override this behavior with meta programming, but got stuck with some odd errors (maybe because of my short Ruby experience), so instead requesting this as a feature. Something like:


data = Scrubyt::Extractor.define do
  fetch url
  set_current_form :form1 # ...or just try to use the first found form as default 
  submit

  ...
end

Comments and changes to this ticket

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

A simple to learn and use, yet powerful web scraping toolkit written in Ruby.

People watching this ticket

Pages