tag:scrubyt.lighthouseapp.com,2008-10-23:/projects/18686-scrubyt/eventsscrubyt / scRUBYt! Events2009-02-27T11:32:01Ztag:scrubyt.lighthouseapp.com,2008-10-23:Event/105537712010-11-24T12:42:44+01:002010-11-24T12:42:45+01:00PradsscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<li>Tag set to <em class="change">“<span class="tag">fail</span>”</em></li>
<li>Milestone order changed from <em class="change">“0”</em> to <em class="change">“0”</em></li>
<div><p>Its surprising that this ticket is almost 2 years old and the
problem still persists. I tried using Scrubyt with various
dependent gems as well as Ruby versions.. still no luck. I'm a Ruby
noob and can't fix all the errors. Since this is such an old
ticket, I'm curious to know what all the previous post authors did?
Is there an alternate solution that you found that just does what
Scrubyt is supposed to be doing? Any help or guidance will be much
appreciated. Thanks!</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/101775022010-11-04T06:11:05+01:002010-11-04T06:11:06+01:00hehescRUBYt!: to_text method no longer seems to work [#34]<div><p><strong><a href="http://www.ed-hardy-jeans-sale.com/">ed hardy
clothes</a></strong><br>
<strong><a href="http://www.ed-hardy-jeans-sale.com/">ed hardy
perfume</a></strong><br>
<strong><a href="http://www.ed-hardy-jeans-sale.com/">ed hardy
shoes</a></strong></p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/83169492010-07-18T06:11:15+02:002010-07-18T06:11:15+02:00jekozyrascRUBYt!: to_text method no longer seems to work [#34]<div><p>Calling this function produces:</p>
<p>undefined method <code>to_text' for :Scrubyt::ScrubytResult
(NoMethodError)</code></p>
<p>Same thing for 'to_csv', which is documented on the wiki</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/83169472010-07-18T06:09:54+02:002010-07-18T06:09:54+02:00jekozyrascRUBYt!: to_text method no longer seems to work [#34]tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73416082010-05-09T00:54:40+02:002010-05-09T00:54:40+02:00cyberkid81scRUBYt!: paging returns an error using next_page - posting workaround [#33]<div><p>When using the next_page function, this error is showing up and
the program exits:<br></p>
<pre>
<code> /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/fetch_action.rb:39:in `restore_host_name': uninitialized class variable @@original_host_name in Scrubyt::FetchAction (NameError)</code>
</pre>
<p>I modified this file:
/usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/fetch_action.rb</p>
<p>I set @@original_host_name to nil and saved. That's it. It
appears fixed. See the code segment below.</p>
<pre>
<code> module FetchAction
@@current_doc_url = nil
@@current_doc_protocol = nil
@@base_dir = nil
@@host_name = nil
@@history = []
@@current_form = nil
@@original_host_name = nil #add this line to fix next_page error</code>
</pre></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73415962010-05-09T00:50:15+02:002010-05-09T00:50:15+02:00cyberkid81scRUBYt!: first tutorial sample program generates run error - help? [#32]<div><p>i see from the code its self that the dumper is no longer
working. i guess i won't use it.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73413012010-05-08T23:34:35+02:002010-05-08T23:34:35+02:00cyberkid81scRUBYt!: first tutorial sample program generates run error - help? [#32]<div><p>Fixed, partially by running these three commands:</p>
<pre>
<code> gem uninstall hpricot mechanize
gem install hpricot -v=0.6.164
gem install mechanize -v=0.8.5</code>
</pre>
<p>I now get an error when the Dumper runs, but the scraping looks
to have worked.</p>
<p>./google.rb</p>
<pre>
<code><result>Ruby Programming Language</result>
<result>Download Ruby</result>
<result>Ruby - The Inspirational Weight Loss Journey on the Style Network ...</result>
<result>Ruby (programming language) - Wikipedia, the free encyclopedia</result>
<result>Ruby - Wikipedia, the free encyclopedia</result>
<result>Ruby on Rails</result>
<result>Ruby&amp;#39;s Diner - rubys.com</result>
<result>Ruby Annotation</result>
<result>Watir</result>
<result>[Ruby-Doc.org: Documenting the Ruby Language]</result>
<result>News for ruby</result>
<result>Books for ruby</result></code>
</pre>
<p>
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:440:in
<code>load_missing_constant': uninitialized constant
Scrubyt::ResultDumper (NameError)</code></p>
<pre>
<code> from /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.5/lib/active_support/dependencies.rb:80:in `const_missing'
from ./google.rb:15</code>
</pre></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73412642010-05-08T23:27:42+02:002010-05-08T23:27:42+02:00cyberkid81scRUBYt!: first tutorial sample program generates run error - help? [#32]<div><p>I followed the tutorial here (and later copy and pasted the code
to see if that helped) <a href=
"http://wiki.scrubyt.org/index.php?title=Your_First_Extractor">http://wiki.scrubyt.org/index.php?title=Your_First_Extractor</a></p>
<p>When running this code:<br></p>
<pre>
<code> require 'rubygems'
require 'scrubyt'
google_data = Scrubyt::Extractor.define do
fetch 'http://www.google.com/ncr'
fill_textfield 'q', 'ruby'
submit
result 'Ruby Programming Language'
end
google_data.to_xml.write($stdout, 1)
Scrubyt::ResultDumper.print_statistics(google_data)</code>
</pre>
<p>I get this error::<br></p>
<pre>
<code>./google.rb
/usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/mechanize.rb:182:in `fill_textfield': undefined method `[]=' for nil:NilClass (NoMethodError)
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/navigation_actions.rb:27:in `eval'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/mechanize.rb:182:in `fill_textfield'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/navigation_actions.rb:27:in `fill_textfield'
from ./google.rb:8
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/shared/extractor.rb:75:in `instance_eval'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/shared/extractor.rb:75:in `initialize'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/shared/extractor.rb:32:in `new'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/shared/extractor.rb:32:in `define'
from ./google.rb:6</code>
</pre>
<p>Does anyone know why that may be happening? This is a fresh
installation so it may not be working 100% properly. The install is
detailed here: <a href=
"http://scrubyt.lighthouseapp.com/projects/18686/tickets/31-jopenssl-error#ticket-31-2">
http://scrubyt.lighthouseapp.com/projects/18686/tickets/31-jopenssl...</a></p>
<p>Thanks!</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73411532010-05-08T22:39:49+02:002010-05-08T22:39:50+02:00cyberkid81scRUBYt!: jopenssl error? [#31]<div><p>To suppress the display of the warnings, I modified the two gems
as detailed below. It looks nicer, but things may still go awry due
to the reasons for the warnings. Does anyone know?</p>
<p>Thanks!</p>
<p>Files I modified to suppress the warning output messages:</p>
<p>
/usr/local/lib/ruby/gems/1.8/gems/nokogiri-1.4.1/lib/nokogiri/version.rb
I commented out line 18 which actually displays the warning and
contains:</p>
<pre>
<code> warn "WARNING: #{warning}"</code>
</pre>
<p>
/usr/local/lib/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb<br>
I added a multi-line comment that starts on line 646, just above
"warn <<eomsg" What I added on line 646:</p>
<pre>
<code> =begin</code>
</pre>
<p>I ended the multi-line comment just after "eomsg" What I added
on line 659:</p>
<pre>
<code> =end</code>
</pre></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73410562010-05-08T22:14:15+02:002010-05-08T22:14:16+02:00cyberkid81scRUBYt!: jopenssl error? [#31]<div><p>Ok. I did some more reading. I got it working, but with two
warnings. The warnings are at the bottom. I have detailed the steps
I took in hopes of saving someone else some time. If you're able to
help with the warnings, they're listed at the bottom of my
post.</p>
<p>Thanks!</p>
<p>I modified ruby-1.8.7/ext/Setup and uncommented openssl. I then
recompiled. That resulted in a new error:<br>
/usr/local/lib/ruby/1.8/openssl/ssl.rb:26: uninitialized constant
OpenSSL::SSL::VERIFY_PEER (NameError)</p>
<p>I did some reading on that and came across this thread: <a href=
"http://redmine.ruby-lang.org/issues/show/863">http://redmine.ruby-lang.org/issues/show/863</a></p>
<p>I modified ruby-1.8.7/ext/openssl/ossl_digest.c to remove the
line containing "rb_require("openssl");" inside of the
"Init_ossl_digest" function (I didn't look at the line
number...just searched for it, found it, and removed it).</p>
<p>I then cd'd back to ruby-1.8.7 and ran the following
commands:<br>
make clean ./configure make make test make install</p>
<p>I then ran this command to see if it would give an error... and
it didn't! :)<br></p>
<h1>irb</h1>
<p>irb(main):001:0> require 'openssl' => true
irb(main):002:0> exit</p>
<p>I tried re-running the google test program... and got another
error....</p>
<p>WARNING: Nokogiri was built against LibXML version 2.6.26, but
has dynamically loaded 2.6.27<br>
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in
<code>gem_original_require': no such file to load -- firewatir
(LoadError)</code></p>
<pre>
<code> from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/firewatir.rb:2
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt.rb:29
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from ./favicon.rb:4</code>
</pre>
<p>So, I ran this:<br>
gem install firewatir</p>
<p>I got this as output:<br>
Successfully installed xml-simple-1.0.12 Successfully installed
json_pure-1.4.3<br>
Successfully installed rubyforge-2.0.4<br>
Successfully installed hoe-2.6.0<br>
Successfully installed s4t-utils-1.0.4<br>
Successfully installed builder-2.1.2<br>
Successfully installed user-choices-1.1.6.1<br>
Successfully installed commonwatir-1.6.5<br>
Successfully installed firewatir-1.6.5<br>
9 gems installed<br>
Installing ri documentation for xml-simple-1.0.12...<br>
Installing ri documentation for json_pure-1.4.3...<br>
Installing ri documentation for rubyforge-2.0.4...<br>
Installing ri documentation for hoe-2.6.0...<br>
Installing ri documentation for s4t-utils-1.0.4...<br>
Installing ri documentation for builder-2.1.2...<br>
ERROR: While generating documentation for builder-2.1.2<br>
... MESSAGE: Unhandled special: Special: type=17, text="<!-- HI
-->" ... RDOC args: --ri --op
/usr/local/lib/ruby/gems/1.8/doc/builder-2.1.2/ri --title Builder
-- Easy XML Building --main README --line-numbers --quiet lib
CHANGES Rakefile README doc/releases/builder-1.2.4.rdoc
doc/releases/builder-2.0.0.rdoc doc/releases/builder-2.1.1.rdoc
--title builder-2.1.2 Documentation (continuing with the rest of
the installation) Installing ri documentation for
user-choices-1.1.6.1...<br>
Installing ri documentation for commonwatir-1.6.5...<br>
Installing ri documentation for firewatir-1.6.5...<br>
Installing RDoc documentation for xml-simple-1.0.12...<br>
Installing RDoc documentation for json_pure-1.4.3...<br>
Installing RDoc documentation for rubyforge-2.0.4...<br>
Installing RDoc documentation for hoe-2.6.0...<br>
Installing RDoc documentation for s4t-utils-1.0.4...<br>
Installing RDoc documentation for builder-2.1.2...<br>
Installing RDoc documentation for user-choices-1.1.6.1...<br>
Installing RDoc documentation for commonwatir-1.6.5...<br>
Installing RDoc documentation for firewatir-1.6.5...</p>
<p>I tried running the google test program again. Guess what? No
run-errors!</p>
<p>root@XXXXXX ~/xpath_playground/ruby]#ruby google.rb<br>
WARNING: Nokogiri was built against LibXML version 2.6.26, but has
dynamically loaded 2.6.27<br>
!!!!! DEPRECATION NOTICE !!!!! The WWW constant is deprecated,
please switch to the new top-level Mechanize<br>
constant. WWW will be removed in Mechanize version 2.0</p>
<p>You've referenced the WWW constant from
/usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/mechanize.rb:17:in
<code>included', please<br/> switch the "WWW" to "Mechanize".
Thanks!</code></p>
<p>Sincerely,</p>
<p>Pew Pew Pew</p>
<p>[{:link_title=>"Ruby Programming Language",
:link_url=>"<a href=
"http://ruby-lang.org/"}">http://ruby-lang.org/"}</a>,
{:link_title=>"Download Ruby", :link_url=>"<a href=
"http://www.ruby-lang.org/en/downloads/"}">http://www.ruby-lang.org/en/downloads/"}</a>,
{:link_title=>"Ruby - The Inspirational Weight Loss Journey on
the Style Network ...", :link_url=>"<a href=
"http://www.mystyle.com/mystyle/shows/ruby/index.jsp"}">http://www.mystyle.com/mystyle/shows/ruby/index.jsp"}</a>,
{:link_title=>"Ruby (programming language) - Wikipedia, the free
encyclopedia", :link_url=>"<a href=
"http://en.wikipedia.org/wiki/Ruby_(programming_language)"}">http://en.wikipedia.org/wiki/Ruby_(programming_language)"}</a>,
{:link_title=>"Ruby - Wikipedia, the free encyclopedia",
:link_url=>"<a href=
"http://en.wikipedia.org/wiki/Ruby"}">http://en.wikipedia.org/wiki/Ruby"}</a>,
{:link_title=>"Ruby on Rails", :link_url=>"<a href=
"http://www.rubyonrails.org/"}">http://www.rubyonrails.org/"}</a>,
{:link_title=>"Ruby's Diner - rubys.com",
:link_url=>"<a href="http://www.rubys.com/"}">http://www.rubys.com/"}</a>,
{:link_title=>"Ruby Annotation", :link_url=>"<a href=
"http://www.w3.org/TR/ruby/"}">http://www.w3.org/TR/ruby/"}</a>,
{:link_title=>"Watir", :link_url=>"<a href=
"http://watir.com/"}">http://watir.com/"}</a>,
{:link_title=>"[Ruby-Doc.org: Documenting the Ruby Language]",
:link_url=>"<a href=
"http://ruby-doc.org/"}">http://ruby-doc.org/"}</a>,
{:link_title=>"El Paso Corporation and Global Infrastructure
Partners Close $1.5 ...", :link_url=>"<a href=
"http://www.marketwatch.com/story/el-paso-corporation-and-global-infrastructure-partners-close-15-billion-ruby-financing-2010-05-05?reflink=MW_news_stmp"}">http://www.marketwatch.com/story/el-paso-corporation-and-global-inf...</a>]
root@XXXXXX ~/xpath_playground/ruby]#</p>
<p>So, I modified this file:
/usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/mechanize.rb
and removed the WWW:: portion of line 17 and re-ran the test
program. It broke. Don't do that. I added back in the WWW:: that I
had removed earlier, re-ran, and got output this time.</p>
<p>I hope this helps somebody besides myself. I've spent far too
much time on getting this just to the point that it runs
post-install. I'm new to ruby, which certainly adds some time for
not knowing what I'm doing per-say...but I've been a sysadmin for
10+ years.</p>
<p>This program - scrubyt - is the only reason I am using ruby on
my system. My new gem list looks like this with everything
functional.</p>
<p><strong><em>LOCAL GEMS</em></strong></p>
<p>actionmailer (2.3.5)<br>
actionpack (2.3.5)<br>
activerecord (2.3.5)<br>
activeresource (2.3.5)<br>
activesupport (2.3.5)<br>
builder (2.1.2)<br>
commonwatir (1.6.5)<br>
ffi (0.6.3)<br>
firewatir (1.6.5)<br>
hoe (2.6.0)<br>
hpricot (0.8.2)<br>
jruby-openssl (0.7)<br>
json_pure (1.4.3)<br>
mechanize (1.0.0)<br>
nokogiri (1.4.1)<br>
openssl-nonblock (0.2.1)<br>
ParseTree (3.0.5)<br>
ParseTreeReloaded (0.0.1)<br>
rack (1.0.1)<br>
rails (2.3.5)<br>
rake (0.8.7)<br>
ruby2ruby (1.2.4)<br>
ruby_parser (2.0.4)<br>
rubyforge (2.0.4)<br>
RubyInline (3.8.4)<br>
RubyInlineAcceleration (0.0.1)<br>
rubyzip (0.9.4)<br>
s4t-utils (1.0.4)<br>
scrubyt (0.4.1)<br>
sexp_processor (3.0.4)<br>
smartcard (0.5.0)<br>
tem_openssl (0.3.6)<br>
tem_ruby (0.16.0)<br>
user-choices (1.1.6.1)<br>
xml-simple (1.0.12)<br>
ZenTest (4.3.1)<br>
zerg_support (0.1.5)</p>
<hr>
<p>Now: Does anyone know how to fix these two notices?</p>
<p>Notice <a href="/projects/18686/tickets/1" title=
"Ticket #1">#1</a>:<br>
WARNING: Nokogiri was built against LibXML version 2.6.26, but has
dynamically loaded 2.6.27<br>
!!!!! DEPRECATION NOTICE !!!!!</p>
<p>Notice <a href="/projects/18686/tickets/2" title=
"Ticket #2">#2</a>:<br>
The WWW constant is deprecated, please switch to the new top-level
Mechanize<br>
constant. WWW will be removed in Mechanize version 2.0</p>
<p>You've referenced the WWW constant from
/usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/navigation/agents/mechanize.rb:17:in
<code>included', please<br/> switch the "WWW" to "Mechanize".
Thanks!</code></p>
<p>Sincerely,</p>
<p>Pew Pew Pew</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/73409802010-05-08T21:36:26+02:002010-05-08T21:36:26+02:00cyberkid81scRUBYt!: jopenssl error? [#31]<div><p>I’m having trouble running your example scripts. I have
searched the forums and have not found an answer. The error I am
presently getting is: no such file to load – jopenssl. Other
information about my system, setup, etc. is below.</p>
<p>Does anyone know how to resolve this? I really want to try this
framework out. I e-mailed Peter directly about this just now but
thought it may be good to post it here as well.</p>
<p>Thanks!</p>
<p>~/xpath_playground/ruby]#ruby google.rb
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in
<code>gem_original_require': no such file to load -- jopenssl
(LoadError)</code></p>
<pre>
<code> from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/local/lib/ruby/gems/1.8/gems/jruby-openssl-0.7/lib/openssl.rb:61
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from /usr/local/lib/ruby/1.8/net/https.rb:102
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/local/lib/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb:2
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
from /usr/local/lib/ruby/gems/1.8/gems/scrubyt-0.4.06/lib/scrubyt.rb:11
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `gem_original_require'
from /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:36:in `require'
from google.rb:2</code>
</pre>
<p>ruby version: ruby 1.8.7 (2008-05-31 patchlevel 0)
[x86_64-linux]<br>
gem version: 1.3.6</p>
<p><strong><em>LOCAL GEMS</em></strong></p>
<p>ffi (0.6.3)<br>
hpricot (0.8.2)<br>
jruby-openssl (0.7)<br>
mechanize (1.0.0)<br>
nokogiri (1.4.1)<br>
openssl-nonblock (0.2.1)<br>
ParseTree (3.0.5)<br>
ParseTreeReloaded (0.0.1)<br>
rake (0.8.7)<br>
ruby2ruby (1.2.4)<br>
ruby_parser (2.0.4)<br>
RubyInline (3.8.4)<br>
RubyInlineAcceleration (0.0.1)<br>
rubyzip (0.9.4)<br>
scrubyt (0.4.1)<br>
sexp_processor (3.0.4)<br>
smartcard (0.5.0)<br>
tem_openssl (0.3.6)<br>
tem_ruby (0.16.0)<br>
ZenTest (4.3.1)<br>
zerg_support (0.1.5)</p>
<p>I’m running CentOS5:<br></p>
<pre>
<code> Linux XXXXXX 2.6.18-164.6.1.el5 [#1](/projects/18686/tickets/1 "Ticket #1") SMP Tue Nov 3 16:12:36 EST 2009 x86_64 x86_64 x86_64 GNU/Linux</code>
</pre>
<p>I installed some ruby rpm’s, but needed a newer version
than what was available in the repos I have configured, so manually
built ruby to the 1.8.7 version (I tried the 1.9x version, but read
that it conflicted with your programs currently).</p>
<p>root@XXXXXX ~/xpath_playground/ruby]#rpm -qa | grep ruby<br>
ruby-irb-1.8.5-5.el5_4.8<br>
ruby-ri-1.8.5-5.el5_4.8<br>
ruby-rpm-1.2.3-1.el5.rf<br>
ruby-libs-1.8.5-5.el5_4.8<br>
ruby-devel-1.8.5-5.el5_4.8<br>
ruby-rrdtool-1.3.8-2.el5.rf<br>
ruby-fam-0.2.0-1.el5.rf<br>
ruby-rdoc-1.8.5-5.el5_4.8<br>
ruby-devel-1.8.5-5.el5_4.8<br>
ruby-tcltk-1.8.5-5.el5_4.8<br>
ruby-shadow-1.4.1-1.el5.rf<br>
ruby-docs-1.8.5-5.el5_4.8<br>
ruby-mode-1.8.5-5.el5_4.8<br>
rubygems-0.9.4-2<br>
ruby-1.8.5-5.el5_4.8<br>
ruby-libs-1.8.5-5.el5_4.8</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/71966712010-04-29T03:18:47+02:002010-04-29T03:18:47+02:00NoelIngrid32scRUBYt!: documentation/tutorials out of sync [#17]<div><p>People in all countries get the <a href=
"http://lowest-rate-loans.com/topics/personal-loans">lowest-rate-loans.com</a>
in different banks, just because that's comfortable and fast.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/71060742010-04-22T16:41:34+02:002010-04-22T16:41:34+02:00panthescRUBYt!: Some problem with scRUBYt! NewPage & fill_textfield... [#30]<div><p>Hi,<br>
I've try scRUBYt! and I think that is very powerful and simple to
learn.<br>
But I've some problem to understand some little things...<br>
Somebody can help me???</p>
<p>1) google_data.to_xml.write($stdout, 1)<br>
this command ever return error<br>
test2.rb:14: undefined method <code>write' for
#<String:0xb69b4194> (NoMethodError)</code></p>
<p>2) If I have two field to fill before a search how I can made
it?</p>
<p>fill_textfield 'Nome', 'Enry'<br>
fill_textfield 'Cognome', 'Shuck'<br>
submit</p>
<p>3) In this code how I can go to the next page?<br>
<a href="http://www.nominando.it/p-9?" title="successiva"><br>
<span class="changing-next">successiva</span> >></a></p>
<p>I've tried this code but don't work:</p>
<p>next_page "successiva", :limit => 5</p>
<p>Thanks<br>
Bye<br>
Luca</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/70405112010-04-18T07:19:41+02:002010-04-18T07:19:41+02:00bobbus1111scRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>+1 Nothing to do to fix this issue ??</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/58437182010-01-14T23:51:26+01:002010-01-14T23:51:27+01:00relixscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<li>Assigned user cleared.</li>
<div><p>I too am still getting errors. I've tried downgrading all gems
to the versions mentioned by Rathgar but it just won't work. Really
sloppy...</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/49629892009-10-22T07:15:50+02:002009-10-22T07:15:50+02:00Chris MaynardscRUBYt!: base_filter.rb determine_example_type regex too narrow [#29]<div><p>This valid XPATH is getting misidentified as a string:<br>
'//div[@class="myclass"]/a/@href' while these work:<br>
'//div[@class="myclass"]/@href' '//div/a/@href' etc.</p>
<p>base_filter.rb line 104 column 102:<br></p>
<pre>
<code> when /^\/{1,2}[a-z]+[0-9]?(\[[0-9]+\])?(\/{1,2}[a-z()]+[0-9]?(\[[0-9]+\])?)*(\[@.+=.+\])?(\/@.+)?$/</code>
</pre>
<p>needs another ? added:<br></p>
<pre>
<code> when /^\/{1,2}[a-z]+[0-9]?(\[[0-9]+\])?(\/{1,2}[a-z()]+[0-9]?(\[[0-9]+\])?)*(\[@.+=.+\])?(\/@?.+)?$/</code>
</pre>
<p>Then the XPATH works fine.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/46156092009-09-21T17:13:47+02:002009-09-21T17:13:47+02:00yoramscRUBYt!: find_form_based_on_tage retruns nil @@current_form [#28]<div><p>gems installed:<br>
scrubyt-0.4.06<br>
mechanize-0.9.3<br>
hpricot-0.8.1-x86-mswin32</p>
<p>I was able to get find_form_based_on_tag to return the correct
@@current_form by substituting the following:<br>
break if
@@current_form.form_node.attributes[lookup_attribute_name].to_s ==
lookup_attribute_value.to_s</p>
<p>for<br>
break if @@current_form.form_node.attributes[lookup_attribute_name]
== lookup_attribute_value</p>
<p>on line 241 of core\navigation\agents\mechanize.rb</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/46087452009-09-20T22:03:30+02:002009-09-20T22:03:30+02:00duboisascRUBYt!: mechanize rb 234 in find_form_based_on_tag undefined method attribute [#27]<div><p>? no idea</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/45721052009-09-16T23:32:53+02:002009-09-16T23:32:53+02:00Audrey LeescRUBYt!: ruby_quiz_189.rb gives exception [#26]<div><p>I have installed the scrubyt demos:<br>
- git clone git://github.com/scrubber/scrubyt_examples.git</p>
<p>I tried this one:</p>
<p>$ ruby ruby_quiz_189.rb
/pt/r1/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/scraping/filters/download_filter.rb:19:in
<code>download_file': undefined method</code>include?' for
nil:NilClass (NoMethodError)</p>
<pre>
<code> from /pt/r1/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/scraping/filters/download_filter.rb:8:in</code>
</pre>
<p><code>evaluate'</code></p>
<pre>
<code> from /pt/r1/lib/ruby/gems/1.8/gems/scrubyt-0.4.1/lib/scrubyt/core/scraping/pattern.rb:250:in</code>
</pre>
<p><code>evaluate'</code></p>
<p>I'd welcome any hints on how to debug ruby_quiz_189.rb so it
will run.</p>
<p>scrubyt depends on some other gems:</p>
<p>hpricot<br>
mechanize</p>
<p>Here is what I have:</p>
<p>$ gem list hpricot</p>
<p><strong><em>LOCAL GEMS</em></strong></p>
<p>hpricot (0.6.164)</p>
<p>$ gem list mechanize</p>
<p><strong><em>LOCAL GEMS</em></strong></p>
<p>mechanize (0.9.3)</p>
<p>Thanks!</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/45213462009-09-11T23:49:45+02:002009-09-11T23:49:45+02:00YurascRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>Sorry for continuing this, but I'm not able to run the following
example:</p>
<pre>
<code class="ruby">require 'rubygems'
require 'scrubyt'
ebay_data = Scrubyt::Extractor.define do
fetch 'http://www.ebay.com/'
fill_textfield 'satitle', 'ipod'
submit
click_link 'Apple iPod'
record do
item_name 'APPLE NEW IPOD MINI 6GB MP3 PLAYER SILVER'
price '$71.99'
end
next_page 'Next >', :limit => 5
end</code>
</pre>
<p>Gems installed:</p>
<pre>
<code class="gems">
builder (2.1.2)
commonwatir (1.6.2)
firewatir (1.6.2)
hoe (2.3.3)
hpricot (0.6.164)
log4r (1.0.5)
mechanize (0.6.3)
nokogiri (1.3.3)
rubyforge (1.0.4)
rubygems-update (1.3.5)
s4t-utils (1.0.4)
scrubyt (0.4.06)
user-choices (1.1.6)
xml-simple (1.0.12)</code>
</pre>
<p>I've tried downgrading of mechanize gem till 0.8.5, 0.7.0 and
finally 0.6.3 versions but there is still an error</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/39841292009-07-21T00:34:54+02:002009-07-21T00:34:54+02:00gcostascRUBYt!: :type => :download doesnt support https [#25]<div><p>I noticed the DownloadFilter class doesnt support https at all
as it is. I edited it to support https, but as you will see it's
not supporting http right now. Please take a look at it.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/39840312009-07-21T00:33:57+02:002009-07-21T00:33:57+02:00gcostascRUBYt!: provide a saving and loading session cookies feature [#24]<div><p>Even though scrubyt deals with cookies from behind the scenes,
it would be really nice if we could save a cookie file after
running a login script and then running another scripts using the
same cookie file.</p>
<p>It could work like this:</p>
<p>@test1 = Scrubyt::Extractor.define :production do</p>
<pre>
<code>... #login stuff goes here</code>
</pre>
<p>end</p>
<p>@test1.save_cookies(cookies_path)</p>
<p>@test2 = Scrubyt::Extractor.define :production, :cookies =>
cookies_path do ... end</p>
<p>Is there a way to do something like this already? I think it
would be a great feature for this API, specially when working with
rails where you have to split the extractions between multiple
controller actions if you want to have any kind of human
interaction.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/38725572009-07-08T12:12:00+02:002009-07-08T12:12:00+02:00llupercalscRUBYt!: Scraping Invasion powerboard [#23]<div><p>Can this be done? I have tried to scrape</p>
<p><a href=
"http://www.911cd.net/forums/">http://www.911cd.net/forums/</a></p>
<p>and</p>
<p>greenenergyinvestors.com/</p>
<p>Each time the fill_field raises an exception. Is this because
there are two forms on the page? I will investigate further later
today.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/35786522009-06-08T14:20:31+02:002009-06-08T14:20:31+02:00sgrayscRUBYt!: Detail pages return duplicate data. [#22]<div><p>I'm copying a post I made on the google groups blog that
describes the issue.</p>
<p>You are having the same problem I reported in "Detail page
problems".<br>
There appears to be a bug in detail page processing. I was able
to<br>
make my examples work by changing detail_page_filter.rb. The change
I<br>
made is a kludge at best however. It does highlight the issue.</p>
<p>Here's what I changed in detail_page_filter.rb.</p>
<pre>
<code> #if @detail_extractor.nil?
# @detail_extractor = Extractor.new</code>
</pre>
<p>@parent_pattern.extractor.mode,
@parent_pattern.referenced_extractor</p>
<pre>
<code> # root_results = @detail_extractor.result
#else
# root_results = @detail_extractor.evaluate_extractor
#end
@detail_extractor = Extractor.new @parent_pattern.extractor.mode, @parent_pattern.referenced_extractor
root_results = @detail_extractor.result</code>
</pre>
<p>This creates a new @detail_extractor each time. There are
problems in<br>
the evaluate_extractor method. When called it returns the
previous<br>
results. I'm not sure why but the method evaluate_extractor in<br>
extractor.rb has a couple of issues when processing detail pages
that<br>
I don't know how to resolve.</p>
<pre>
<code> catch :quit_next_page_loop do
loop do
url = get_current_doc_url #TODO need absolute address here</code>
</pre>
<p>2/4<br></p>
<pre>
<code> @processed_pages << url
@root_patterns.each do |root_pattern|
@root_results.push(*root_pattern.evaluate(get_hpricot_doc,</code>
</pre>
<p>nil))<br></p>
<pre>
<code> end</code>
</pre>
<p>The line "url = get_current_doc_url" always returns nil. The
line<br>
"@root_results.push(*root_pattern.evaluate(get_hpricot_doc, nil))"
is not executed. I falls out of the loop without doing anything
and<br>
returns the previous results.</p>
<p>Scott</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/35283782009-06-02T18:48:42+02:002009-06-02T18:48:42+02:00orisunscRUBYt!: 2 forms in a page [#21]<div><p>Hi,<br>
during my experiences with scrybt I faced a problem with a web page
with two forms. Submit uses the last form.</p>
<p>HOw is it possible to submit the first one or how can I control
this problem. I tried it with submit (index), it doesn't help!<br>
I tried like this:<br>
submit
"/html/body/table/tbody/tr[4]/td[2]/div/div/div[3]/form[1]"<br>
It is the same, the second form is the same.<br>
In the forms there is not any text_field to be filled.<br>
I attach a copy of the html page and my ruby code.</p>
<p>I would like understand how can I control this to work with this
tool.</p>
<p>Thanks for help...<br>
orisun</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/35063592009-05-30T12:03:28+02:002009-05-30T12:03:28+02:00orisunscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>It works..<br>
scrubyt (0.4.06) let it work...</p>
<p>good work... nice tool..<br>
thx</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/35063582009-05-30T11:58:43+02:002009-05-30T11:58:43+02:00orisunscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>additional..<br>
Excuse me for the unformatted code (I copied it from gedit) and<br>
Thanks for the helpful tool...</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/35063122009-05-30T11:35:20+02:002009-05-30T11:35:20+02:00orisunscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>Hi,<br>
I tried the example:<br>
google_data = Scrubyt::Extractor.define do<br>
fetch '<a href=
"http://www.google.com/ncr">http://www.google.com/ncr</a>'
fill_textfield 'q', 'ruby' submit<br>
result 'Ruby Programming Language' end<br>
google_data.to_xml.write($stdout, 1)<br>
Scrubyt::ResultDumper.print_statistics(google_data)</p>
<p>And I still get the error:<br></p>
<blockquote>
<p>undefined method <code>write' for #<String:0xb7551a88>
(NoMethodError)</code></p>
</blockquote>
<p>My gems:<br>
firewatir (1.6.2)<br>
hpricot (0.6)<br>
mechanize (0.8.5)<br>
scrubber-scrubyt (0.4.30)</p>
<p>I tried different versions of the gems, but nothing helps.</p>
<p>orisun</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/29547142009-04-01T15:57:48+02:002009-04-01T15:57:48+02:00xds2000scRUBYt!: firewatir gem required, even though docs say it isn't [#20]<div><p>same here,i think the tutorial is detail.so let me confused.</p></div>tag:scrubyt.lighthouseapp.com,2008-10-23:Event/26123382009-02-27T12:32:01+01:002009-02-27T12:32:01+01:00RathgarscRUBYt!: Latest mechanize (0.9.0) breaks ScrubyT(0.4.1) [#14]<div><p>Same here.</p>
<p>Installed all latest gems from fresh and fails. Un-installed
latest Mechanize and installed 0.8.5 and works fine. No other gems
altered.</p>
<p>All related gems installed (when working):</p>
<pre><code>
firewatir (1.6.2)
hoe (1.8.3)
hpricot (0.6.164)
mechanize (0.8.5)
nokogiri (1.2.1)
rails (2.3.0, 2.2.2, 2.1.0)
rubyforge (1.0.2)
scrubyt (0.4.06)
xml-simple (1.0.11)
</code></pre></div>