#13 open
Jonas Grimfelt

Not supported: Forms without input fields

Reported by Jonas Grimfelt | January 10th, 2009 @ 01:35 PM

In one of my projects I came across that Scrubyt don't support submitting forms without text/select input fields, which is a more common scenario than what one might think.


Page: zshare.com/download/xxxx

<form action="" method="post" name="form1">
  <input id="referer2" type="hidden" value="" name="referer2"/>
  <input id="download" type="hidden" value="1" name="download"/>
  <input width="219" type="image" height="57" border="0" src="/images/download.gif" name="imageField"/>

It seems by using instruction such as:

fill_textfield 'user_id', 'foo'

is required to make Scrubyt set a current form, i.e. @@current_form in Scrubyt::Navigation::Mechanize - otherwise it is nil and raises an exception. I consider this as a bug at this point, but quite easy one to solve.

Code that (of course) raise error:

data = Scrubyt::Extractor.define do
  fetch url



Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/agents/mechanize.rb:71:in submit': uninitialized class variable @@current_form in Scrubyt::Navigation::Mechanize (NameError)

from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/navigation/navigation_actions.rb:58:in `submit'
from zdown.rb:79:in `scrape_download_url'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `new'
from /Library/Ruby/Gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt/core/shared/extractor.rb:32:in `define'

I tried to override this behavior with meta programming, but got stuck with some odd errors (maybe because of my short Ruby experience), so instead requesting this as a feature. Something like:

data = Scrubyt::Extractor.define do
  fetch url
  set_current_form :form1 # ...or just try to use the first found form as default 


Comments and changes to this ticket

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

A simple to learn and use, yet powerful web scraping toolkit written in Ruby.

People watching this ticket