Search Engines Optimize

[David]

Just so you don't think we are ignoring you, we have started working on a resolution to this issue for the next release.

[Anguz]

thank you David

btw, I found how to write the regex for the renaming, but I can only rename complete urls so far, not just portions... is there a way to do this? like replacing "adisplay" with "action=display" no matter where in the url it is?

I ask this because the url format varies a lot depending on the like and it's too much work to write a rule for each format... does anyone know how to do this? thx!

[David]

The following code allows you to change your urls to be like
index.php?thread=4392

We think this will make Google index it, since it only puts one variable in the url.  Feel free to take this code and make an actual mod or just edit your source files to change links to use this format.

In index.php
Before
   $actionArray = array(
   
Add

Code Select Expand


   if(isset($thread))
   {
      include_once("$sourcedir/Display.php");
      $result = mysql_query('SELECT ID_BOARD FROM '.$db_prefix.'topics WHERE ID_TOPIC='.$thread);
      $row = mysql_fetch_row($result);
      global $currentboard, $threadid, $board;
      $currentboard = $row[0];
      $board = $row[0];
      $threadid = $thread;
      $action = 'display';
      LoadBoard();
   }

[Anguz]

hmm... does that code load the proper thread even if you only have $threadid in the url? no $action, no $board, no $start?

if the url will always be index.php?threadid=4392 (or whatever number), then I can write the regex to rename the url much faster...

I can only think of one problem... when the thread has multiple pages, then the links that will require that $start variable won't be accessed, only the first page will

so maybe that can be modified to have $threadid and $start

as I see it, there's other variables needed as well, cause to get to a thread, the spider has to get by the boards index and then the multiple pages of each board

so the needed variables will be $action ("display" and "messageindex"), $board, $threadid and $start

if the url could always only have those four, even if it's a null value, then the format would remain the same and only one renaming rule will be needed

of course there's other variables needed, but with those it'll suffice to have the spiders index all the messages in every page/thread/board...

it doesn't matter (search engine wise) if the others urls aren't friendly yet, you don't need the bot trying to post a reply with quotes from every message, send messages, etc... actually, not having the bot do that, will save bandwidth

so having those 4 values in the url, always in the same order, will make the boards indexable and save bandwidth at the same time... nice solution

the url should be outputted in the friendly format in every link as well... a good format I can think of is:

http://www.yabbse.org/community-d-141-23503-new.htm

which is renamed to:

http://www.yabbse.org/community/index.php?board=141;action=display;threadid=23503;start=new

what would be the code needed to always output the url in that format even if there's no $start or $action or $threadid? in such cases, the default value for those variables could be passed

[[Unknown]]

Alright, here's my solution...

Add this where David talked about:

Code Select Expand

  if (isset($thread))
  {
     global $currentboard, $threadid, $board, $start, $action;

     $result = mysql_query("
        SELECT ID_BOARD
        FROM {$db_prefix}topics
        WHERE ID_TOPIC= " . (int) $thread . "
        LIMIT 1") or database_error(__FILE__, __LINE__);
     list ($board) = mysql_fetch_row($result);
     mysql_free_result($result);

     $currentboard = $board;
     $threadid = (int) $thread;
     if (strpos($thread, '/') !== false)
        $start = substr($thread, strpos($thread, '/') + 1);
     $action = 'display';
     LoadBoard();
  }

Alright... so far so good.  To test this, try index.php?thread=1/new I think that should work.

Now, you'll need to add the following after this in index.php:
$chklngfile = $chklngfile2 = $settings[23];

Code Select Expand

ob_start('ob_fix_thread_url');

function ob_fix_thread_url($buffer)
{
  global $scripturl;

  return preg_replace(
     array('~"' . preg_quote($scripturl, '~') . '?board=\d+[;&]action=display[;&]threadid=(\d+)"~', '~"' . preg_quote($scripturl, '~') . '?board=\d+[;&]action=display[;&]threadid=(\d+)[;&]start=(\d+)"~'),
   array('"' . $scripturl . '?thread=\\1"', '"' . $scripturl . '?thread=\\1/\\2"'),
   $buffer);
}

I can't promise this will work, but it should.  Please tell me how it goes...

-[Unknown]

[[Unknown]]

Here is some fixed code:

Code Select Expand

   if (isset($thread))
   {
      global $currentboard, $threadid, $board, $start, $action, $cgi, $scripturl;

      $result = mysql_query("
         SELECT ID_BOARD
         FROM {$db_prefix}topics
         WHERE ID_TOPIC= " . (int) $thread . "
         LIMIT 1") or database_error(__FILE__, __LINE__);
      list ($board) = mysql_fetch_row($result);
      mysql_free_result($result);

      $currentboard = $board;
      $threadid = (int) $thread;
      if (strpos($thread, '/') !== false)
         $start = substr($thread, strpos($thread, '/') + 1);
      $action = 'display';
      LoadBoard();
      $cgi = $scripturl . '?board=' . $board;
   }

Code Select Expand

ob_start('ob_fix_thread_url');

function ob_fix_thread_url($buffer)
{
   global $scripturl;

   return preg_replace(
      array('~"' . preg_quote($scripturl, '~') . '\?board=\d+[;&]action=display[;&]threadid=(\d+)"~', '~"' . preg_quote($scripturl, '~') . '\?board=\d+[;&]action=display[;&]threadid=(\d+)[;&]start=([^;&]+(?:[;&]boardseen=1)?)"~'),
      array('"' . $scripturl . '?thread=\\1"', '"' . $scripturl . '?thread=\\1/\\2"'),
      $buffer);
}

And that second section of code should be _AFTER_:

Code Select Expand

else
   ob_start();

-[Unknown]

[Surfy]

 i dont know what should be the Problem with the searchengines...

Yahoo & Google have indexed my Board well, google is daily on the Board, mostly with 2 IPs.

They scan deeper than the first Page on each Board. Also the flood my errorlog, when they try to view Profiles, Memberlist and so on  

And i have done nothing more, than to use "Add URL" on some searchengines....

The Community wasnt very happy about this, many many new People asked basic questions about Keythemes from the Board. In Google the Keyhemes was long times on Place 1 from some searchterms, on yahoo on place three... Now im glad that the situation is normalized..

Surfy

[I, Brian]

[unknown] -

The corrected code nearly works - the URLs do change - but end up calling the index page.  

I've tried a few times and the same thing happens each time, so hopefully it isn't user error.

Great progress though - sincere thanks for the effort going into this.


Surfy -

I've no idea what you're doing, but something doesn't sound right. Search engines - especially Google - should be able to access member profiles without issue, so I don't know why they should throw up errors. What are the IPs of Googlebots is it that are actually visiting, if you don't mind my asking? Do you have a link to your board as well?

[Surfy]

[unknown] -

Surfy -

I've no idea what you're doing, but something doesn't sound right. Search engines - especially Google - should be able to access member profiles without issue, so I don't know why they should throw up errors. What are the IPs of Googlebots is it that are actually visiting, if you don't mind my asking? Do you have a link to your board as well?

On my Board, Guests cant view Memberprofiles  

This Bots has today produce many ( errors   :

66.196.72.37 INKTOMI / Yahoo
66.196.72.35 INKTOMI / Yahoo
66.196.72.23 INKTOMI / Yahoo

Only from today. In WhoisOnline i see the threads they Scan, they Scan deeper than only the topics of each Board, realy no problems with that.

Now, i have testet it again. In Fact the eniges Scan the Boardindex, the Messageindex of each Board, and the Stuff inside the Post.

Actually i dont found something older from Page >1   in Google...

Maybee its that what you discuss here...

Surfy

[Surfy]

Current are 4 Crawler IPs Online:

Guest (209.237.238.175)
Guest (209.237.238.173)
Guest (209.237.238.174)
Guest (66.196.72.110)

   

Unknown, ill test your way soon, maybee i could get them out of the Board. No errorloggentry`s if they use the Static Pages.....

But what is with "hidden" Boardareas & your mod? They generate it anyway?


Surfy

[[Unknown]]

I, Biran....

If you'll notice.... THIS FORUM is now using my code.  No joke, and it works fine.

Email me your index.php and I'll add the code to it in the correct places.

-[Unknown]

[I, Brian]

Surfy -

It sounds like your getting good attention, so at least you should do well with a fully indexed board.

As for errors - I often forget that some people disallow guests.  



[unknown] -

I think I may have to - I'm not sure what I'm doing wrong.

First section of code after:

Code Select Expand

   ob_start("ob_gzhandler");

else

   ob_start();

and second section before:

Code Select Expand

$actionArray = array(

??

[[Unknown]]

No, you have them reversed.  Second section before the ob stuff, first section before the action stuff.

-[Unknown]

[I, Brian]

Yes - sorry, I stated incorrectly in the previous post. I had them both originally the way required.

Not working for myself, but as it works here then it must be user error on my part. I'm not sure what the issue is - probably something simple and silly that'll take a while to tease out. I'll try and figure it out.

[I, Brian]

Hi unknown -

I've run two complete fresh installs of 1.5.4 - safe-mode install, and install through the .ya file. Even on those occassions I couldn't get the changed URL to work properly, even with the index e-mailed - still the threads show as a single variable, but a click through sends the user straight back to the board index.

I figure there's nothing in the code that would be specific to any server OS. And my side .htaccess files are empty, so it's not that either. So I'm stumped.

Is it possible there was an overlooked tweak in the index.php file code for this site, when applied?