DySE™ make-lib : Revisions History  
Home > DySE™ make-lib > Revisions History   Updated: 1-Jun-2010

Revisions History

Browse Revisions History

v11.08.22 (August 22, 2011)

v10.08.04 (August 4, 2010)

  1. New Feature: Downloadable datafeed fixes.

    If available, datafeed fixes are now automatically downloaded as a separate fixes.txt file. This separation of datafeed fixes from the make.pl file allows new datafeed fixes to be released without having to install a new make.pl file. To disable automatic downloading of fixes, set new configuration variable fixes.download to no. If downloading is disabled or fails, a default fixes.txt file is created.

  2. Improvement: Download process tests for 302/404 errors.

    The download process now checks to see if a "302 File Moved" or "404 File Not Found" error occurred and reports a warning: "status: File moved; not downloaded" or "status: File not found", respectively.

(intermediate versions to be documented)

v10.06.01 (June 1, 2010)

  1. Fix: All datafeed lines counted even if contains Ctrl-Z.

    Before starting Pass 1, make.pl quickly counts the number of lines in the datafeed file. Previously, if the datafeed contained a Ctrl-Z character, the counting would stop at that line in the datafeed (Ctrl-Z is an end-of-file marker). All lines are now counted.

v10.02.13 (Feb. 13, 2010)

  1. New Feature: Internet caching added.

    Added support for caching of internet requests. Cache files stored in /cgi-bin/dyse/merchant/data/cache/. Cache files are compressed if perl supports Compress::Zlib comprssion. If you wish to save disk space, you can delete cache contents after running make.pl

v9.10.29 (October 29, 2009)

  1. Improvement: Checks for invalid Datafeed.

    If the downloaded datafeed file appears to be an HTML file then a fatal error is reported. On one occassion, a merchant did not upload their datafeed file to their website, and when DySE tried to access it, the merchant's website redirected to their "404 File Not Found" page which DySE started to process as if it was the datafeed.

  2. Improvement: Percentage completed shown during datafeed download.

    During a datafeed download, the percentage completed so far (i.e.: "10%") is shown.

  3. New Feature: Offline URL in offline.txt

    The default affiliate link (i.e.: link to the merchant's homepage, with your affiliate ID) is now written to the file offline.txt in the directory where make.pl is located. Thus while make.pl is running, people visiting your website will be redirected to the URL found in offline.txt

    Note: Requires view.pl higher than v9.01.29; as of this release, feature is only available in the latest view.pl beta version.

v9.07.07 (July 7, 2009)

  1. Minor internal revision. New configuration variables (default yes): write.cache.cat, write.cache.item, write.cache.sitemap.html, write.cache.sitemap.google, write.cache.ssi. These new configruation variables are currently not used by view.pl v9.01.29 (or lower). They are intended to be used when a future view.pl supports HTML page caching.

v9.01.29 (January 29, 2009)

  1. New Feature: '-delimeted configuration values ('value')

    Configuration variable values (in make-ini.txt) that contain space characters can now be delimeted with apostrophe characters (e.g.: variableName = 'My Home Page'). Thus now supports: 'value', "value", and [value] for specifying a value.

  2. Improvement: Clarified warning message when key file is invalid.

    When a key file is found but it is invalid (e.g.: due to being incorrectly uploaded), the warning message is now "!!! WARNING: Key > valid key not found; running in demo mode" (the word "valid" was added).

  3. Improvement: Open/Closing quotation marks translated to "

    Open/Closing quotation marks (&#147 and &#8220 / &#148 and &#8221) in the datafeed are converted to the simple quotation character (").

v8.11.14 (November 14, 2008)

  1. Improvement: Human-Readable Sitemap generated faster.

    Human-readable sitemaps (at /sitemap/) are optimized and are now generated faster. Requires view.pl v8.11.14 (or higher) to take advantage of this optimization.

  2. New Feature: Time Checkpoints added to log file.

    Time Checkpoints added to log file so can see how much time elapsed between checkpoints and since start of program.

v8.04.13 (April 13, 2008)

  1. Improvement: Additional log headers added

    To improve the readability of the log output, two additional section headers have been added: "key file validation" and "diagnostics".

  2. Improvement: Handles DOCUMENT_ROOT with extraneous trailing /

    On some servers, the server's DOCUMENT_ROOT CGI environment variable has an extraneous trailing slash (/) character. The character is now removed.

v8.02.23 (February 23, 2008)

  1. New Feature: Configuration files written to log file.

    Configuration files are written to the log file for easier debugging.

  2. New Feature: Version check can be disabled.

    New configuration variable version.check specifies whether make.pl should check the version number against the latest version number available at our c3scripts.com website. Set to yes (default) to check version; set to no to disable version check. If your server is unable to access c3scripts.com for some reason, then set version.check to no.

  3. Improvement: make.pl version check saves version to make-version.txt

    When make.pl checks the latest version number available, it is saved locally to make-version.txt

v7.12.11 (December 11, 2007)

  1. New Feature: New {merch.*} Fields Supported.

    New {merch.*} template fields are supported. Specifically:

    • {merch.desc} : brief description of merchant's products (e.g.: "Clocks"), or empty if not defined.
    • {merch.fulldesc} : long description of merchant's products (e.g.: "Clocks, Mantel Clocks, Anniversary Clocks, Atomic Clocks, Outdoor Clocks, ..."), or empty if not defined.
    • {merch.about} : URL of merchant's "About Us" page, or home page URL if no such page; link contains your affiliate ID.
    • {merch.contact} : URL of merchant's "Contact Us" page, or home page URL if no such page; link contains your affiliate ID.
    • {merch.help} : URL of merchant's "Customer Support Help" page, or home page URL if no such page; link contains your affiliate ID.
  2. Improvement: Sitemaps display faster.

    Sitemaps now display faster; sitemap is sorted once and stored by make.pl rather than sorted each time by view.pl. Advanced: Was O(n log n), now O(1).

  3. Improvement: Pagebar separator changed to · (dot).

    Default value of pagebar.separator has been changed to "·" (shows as ·) instead of "|" (vertical bar) so that the pagebar now looks like: pages: 1 · 2 · 3 · 4 · 5 · 6> · next»

  4. Improvement: No longer uses www.kqzyfj.com links for cj.com merchants.

    For cj.com merchants (AquinasAndMore.com and InstrumentPro.com), the domain name www.kqzyfj.com is no longer used in links. That domain belongs to cj.com and is one of several domains that cj.com uses in links. The 3rd-party anti-spyware utility program SpyBot S&D now blocks access to that domain; therefore, DySE no longer uses that domain in links.

v7.09.26 (September 26, 2007)

  1. New Feature: data.compress configuration variable. New configuration variable data.compress specifies whether to use compression when writing to the database file. Written data is compressed when set to yes (default), and is written uncompressed when set to no. Useful when you run make.pl on your Windows PC and upload the data.db and cfg.db files to your server that does not support compression; in that case, set data.compress to no

  2. Improvement: Improved "file not found" handling. In the event that the merchant moves a sub-category, its URL changes. This version of make.pl makes it easier for view.pl to find the new URL. A version of view.pl higher than v7.06.22 is required to take advantage of this improvement.

  3. Improvement: make.pl stops if empty datafeed file. If the merchant's datafeed file is empty or missing, the make process stops and the local database file is left as is.

  4. Improvement: make.data configuration variable. New configuration variable make.data specifies whether to build the data.db database file or always leave it as is. If make.data is set to no then the database file is always left as is. The default of yes causes the database file to be conditionally built as usual. This configuration variable is particularly useful if you have more than one installation (on the same server) and are sharing one database among them and only one installation is to do the build process.

v7.06.22 (June 22, 2007)

  1. Improvement: Google sitemaps faster. Google sitemaps are now created faster. Previously, very large Google sitemaps might take too long to generate and Google.com spider would timeout thus causing a non-fatal error to appear in Google's spider log. Note: This improvement only works for view.pl that is v7.06.22 or higher. Advanced: Went from O(n^2) to O(n log n).

  2. Improvement: Invalid database file detected. If the database file (data.db) is invalid, it is automatically overwritten when make.pl is run. Previously, make.pl would report an error and stop.

  3. Improvement: data.nocheck defaults to 1 hour. Configuration variable data.nocheck sets the minimum number of hours that must pass before make.pl will check the merchant's website for a new datafeed. The default for data.nocheck is now 1 hour, previously the default was 24 hours. Note: You must still run make.pl for it to check for a new datafeed. If you run make.pl over and over again, it will check for a new datafeed at most once in 1 hour (or however many hours you have specified in data.nocheck).

  4. Fix: Duplicate items filtered out. For some merchants, a product might appear more than once in the same category. Now each product appears only once in each category (a product can still appear in multiple categories if it is in multiple categories in the datafeed).

v6.06.30 (June 30, 2006)

  1. Improvement: Database write failures detected immediately. If writing to the database fails (e.g.: because disk space has become full), an error is now reported immediately. Previously, the write error was undetected (leaving the database in an inconsistent state) and most likely a data inconsistency error would be reported on a subsequent read operation.

  2. Improvement: Supports DySE::Kitchen. Contains internal changes to support DySE::Kitchen.

v6.06.13 (June 13, 2006)

  1. Fix: Database rebuilt when key installed. With DySE™ scripts using Time Shared ID, the database is now rebuilt when the key is installed and make.pl is run. Previously, if make.pl was run without a key, and a key was uploaded, and make.pl run again, then the database would not be rebuild thus causing no ID to appear in the "buy" links. This fix detects the change in ID and causes the database to be rebuilt when make.pl is run again. (Advanced: The build signature was only recording cfg.id).

v6.06.10 (June 10, 2006)

  1. New Feature: ID Time Sharing support added. Support for ID Time Sharing added. This feature enables us to create DySE™ modules for merchants that have 1-tier commission structures.

  2. Improvement: make2.pl version checked. The version of make2.pl is checked and must be at least v6.06.10 otherwise it is not used. See below: "URL's of similarly named items fixed".

  3. Improvement: Rebuilds database after changing "test" configuration variable. If you change the test configuration variable (specifies how many datafeed rows to process during testing) and then run make.pl, the database is now always rebuilt. Previously, make.pl might report "Data > data.db file is already up-to-date with datafeed; leaving data.db as is."

  4. Fix: URL's of similarly named items fixed. If all the items have a common prefix, that is, they all start with the same set of characters (e.g.: "Toronto Blue Jays", "Toronto Argonauts", "Toronto ..."), the filenames now include the prefix. Previously the prefix would not appear in the URL's. This bug has also been fixed in make2.pl; this fix is why the make2.pl version number is now checked (see above).

v6.04.08 (April 8, 2006)

  1. New Feature: Root Category Selection. New configuration variable category.root enables selection of the root category (default is "", meaning all catgories). For example, with DySE::InstrumentPro, setting it to "Guitars" cause only the products in the "Guitars" category to be included in the store. This feature enables you to create niche stores. See Root Category Selection for details.

  2. New Feature: Checks for minimum version of view.pl. If make.pl requires a mimimum version of view.pl, make.pl reports a fatal error if view.pl is below the minimum.

  3. Improvement: Rebuilds database after upgrading make.pl. If you upgrade make.pl and then run make.pl, the database is now always rebuilt. Previously, make.pl might report "Data > data.db file is already up-to-date with datafeed; leaving data.db as is."

  4. Improvement: Leftover .tmp files deleted. Before downloading the datafeed, any leftover .tmp files (older than 1 hour) are deleted first.

  5. Improvement: UTF-8 character processing optimized. Processing of UTF-8 characters in the datafeed has been optimized to run faster.

  6. Improvement: Handling of quotes in datafeed improved. Handling of quotation marks in the datafeed has been improved. Some records with incorrectly placed quotation marks were previously discarded. Those records are now included. (Advanced: x"x becomes x""x).

  7. Fix: Constant values recognized correctly. To reduce space in the database, if a field has a constant value for all products (e.g.: currency is always "US$"), then only one copy is stored in the database and it is shared by all products. This feature is now working correctly. Previously, if the value of the first product and second product were the same, then the field was considered to be constant even if other products had a different value in that field, and thus the product pages would show a constant value for that field.

v6.01.29 (January 29, 2006)

  1. Improvement: Minor improvements.

    1. Displays "Pass 1 of 2:" and "Pass 2 of 2:" while processing products rather than just "Processing:".

  2. Fix: Minor fixes.

    1. Products with malformed category hierarchy (e.g.: "Books > > Fiction") are discarded.
    2. Trailing empty categories are truncated (e.g.: "Books > Fiction > " becomes "Books > Fiction") and product is processed.
    3. Product not added to category if category already has 100,000 products or categoriy's item ID list is 1 millilion characters.
    4. Prices chopped to 2-decimal places (e.g.: "1.2500" becomes "1.25").

v5.12.03 (December 3, 2005)

  1. Fix: make.pl works from cron job. On some servers when make.pl is run from a cron job, a fatal error would be reported: "Make > unrecognized data from {path}/make2.pl sub-program: ERROR ./make2.ini: couldn't open for reading: No such file or directory". This version fixes the problem. (Technical: The cron job didn't change to the proper directory thus make2.ini is not in the current directory; make.pl now tells make2.pl what directory to use).

v5.09.22 (September 22, 2005)

  1. Improvement: Reports fatal error if duplicate SKU's. If duplicate SKU's are found in the datafeed file (i.e.: more than one datafeed record with the same SKU), then a fatal error is reported.

v5.07.22 (July 22, 2005)

  1. Improvement: Category name sort improved. When sorting category names, category names starting with the word "By" are sorted at the end (e.g.: "By Manufacturer"). Thus /go/SKU quick-links go to the product at its standard category location in the category tree rather than at its "By" location in the category tree.

v5.07.06 (July 6, 2005)

  1. New Feature: Cloaking configuration variables added. Configuration variables links.cloak (default: no) and links.cloak.spiders (default: bot) have been added. Used by DySE™ view.pl v5.07.06 (or higher).

    If links.cloak is yes, then all product buy links become http://YourDomain.com/buy/SKU rather than linking directly to the merchant's website. When the user clicks on the link, view.pl redirects the user to the final URL so the user ends up at the product page at the merchant's website.

    If the user-agent of the /buy/SKU request contains the word "bot" (specified by links.cloak.spiders) then view.pl redirects /buy/SKU links back to the product page at your website. Thus spiders won't see that product links lead off of your website. Advanced: links.cloak.spiders is used as the Perl regular expression in the case-insensitive comparison; you could set links.cloak.spiders to a complex regular expression "string|string|string|..." such as "googlebot|msn|teoma|slurp".

v5.06.29 (June 29, 2005)

  1. Improvement: DOCUMENT_ROOT value reported. The server setting DOCUMENT_ROOT is now reported in the log. This setting indicates the disk path of your website's root directory (e.g.: /home/virtual/c3scripts.com/var/www/html/). This setting is used by make.pl and view.pl to configure the dyse images directory. Some servers have DOCUMENT_ROOT set incorrectly!

  2. Improvement: Redirects to moved webpages. If a product moves to a different directory (e.g.: category name change or product moved into a sub-category), DySE™ view.pl redirects to the new URL. This feature only works if the name of the product does not change (i.e.: the last part of the URL does not change) and view.pl is v5.06.22 or higher.

v5.05.31 (May 31, 2005)

  1. Fix: Checks /dyse directory for broken links. When checking for broken symbolic links in /dyse directory, now also checks those links that have digits in the link name; previously only checked those links that were all letters. Also, broken symbolic links are now correctly deleted; previously they were not deleted.

v5.05.25 (May 25, 2005)

  1. New Feature: Checks if graphics uploaded in ASCII mode. If the graphics in the templates directory were uploaded using ASCII mode rather than binary mode, a warning is displayed saying that the graphics should be reuploaded using binary transfer mode (since graphic files are binary files!). If graphics are uploaded using ASCII transfer mode, they will be damaged and will not display properly.

  2. New Feature: Checks /dyse directory for broken links. The /dyse directory contains symbolic links to the templates directory of each merchant. If the symolic link is broken (points to the wrong directory), then template graphics will appear broken ([x]). The make.pl script now checks for and deletes broken symbolic links; if you then view any page of the DySE™ store, view.pl will recreate the correct symbolic link.

v5.05.18 (May 18, 2005)

  1. Improvement: Sort product names with numbers. When the products are sorted by name, the sort algorithm now takes into account any numbers that appear in the name so that the numbers are sorted as number rather than characters. For example, "Cable (6 feet)" now comes before "Cable (12 feet)" since the number 6 is less than the number 12. Previously, the sort would have put "Cable (12 feet)" before "Cable (6 feet)" since the character "1" is less than the character "6".

  2. Improvement: Stop if no products. Stops processing after pass 1 if no products found (likely due to change in format of datafeed file).

  3. Fix: µ handled in product names. The charcter "µ" in product names (e.g.: 45µm cable) is now converted to the letter "u" when determining the filename of the product (e.g.: 45um-cable.html). Previously, the "µ" was unrecognized and converted to a dash (e.g.: 45-m-cable.html)

  4. Fix: Prices with missing cents formatted correctly. If the product price is missing the ones cents digit, a "0" zero digit is appended to the price. For example, "$1.5" becomes "$1.50".

v5.05.11 (May 11, 2005)

  1. Improvement: Total datafeed download size displayed. At the start of datafeed downloading, the total datafeed size is displayed.

  2. Improvement: Time remaining displayed. During pass 1 and pass 2, the estimated time remaining to complete that pass is displayed.

  3. Fix: Can run make.pl via cron job. You can now run make.pl via a cron job without having to first do a "cd" command to change to the directory where make.pl is located. The directory is now determined automatically and thus a batch file that first does a "cd" command and then calls make.pl is no longer required. This fix simplifies how to run make.pl via a cron job.

  4. Fix: Constant values recognized correctly. To reduce space in the database, if a field has a constant value for all products (e.g.: currency is always "US$"), then only one copy is stored in the database and it is shared by all products. This feature is now working correctly. Previously, if the value of the first product was empty, then it was assumed that the field was constantly empty even if other products had a different value in that field.

  5. Fix: Keeps running without LWP. If the perl module LWP (used for internet access) is not installed, then make.pl no longer tries to access www.c3scripts.com to see if a newer version of make.pl is available. Previously, make.pl would try to access www.c3scripts.com and would crash since LWP was not available.

  6. Fix: Degrees symbol (º) now displayed properly. The degrees symbol (e.g.: 45º) is now displayed properly. Previously, it would show up as ° (e.g.: 45°) which is the HTML encoding of the degrees symbol.

  7. Fix: &#number; now displayed properly. Numerically encoded HTML characters (e.g.: ‹) are now handled properly. Previously, the # would be removed (e.g.: ‹ became &139;) thus breaking the encoding. (This bug only applied to values less than 160 or greater than 255). (FYI, ‹ is ).

v5.04.08 (April 8, 2005)

  1. New Feature: Quick links added. Can now use a "quick link" SKU-based URL to refer to a web page rather than using its long URL. The format is "http://YourDomain.com/go/SKU".

    For example, http://www.gifts-finder.com/go/f-28087 is the "quick link" for http://www.gifts-finder.com/set1/angels/alabaster-angel-gifts/alabaster-angel-kneeling.html

    Users can type in the "quick link" URL to get to the product. You can also use "quick link" URL's in an email newsletter.

    You can use any word in place of "go" such as "jump". The only special word is "buy". If you use "buy", such as http://www.gifts-finder.com/buy/f-28087 then the user is redirected to that product at the merchant's website. If the SKU is unknown, the user is redirected to your store home page at your website.

    Note: Requires DySE™ vew.pl v5.04.08. To display the quick link URL on your items page, add {item.quick.go} or {item.quick.buy} to your items.html template file where you want the URL to appear.

  2. New Feature: Checks version of view.pl. Checks version of view.pl and reports warning if newer version available.

  3. Improvement: Can run make.pl via cron job. Can run make.pl via a cron job without having to first do a cd (change directory) command to the directory where make.pl is located.

  4. Improvement: Interrupted make.pl is recognized. If make.pl is interrupted, the incomplete data.db database file is recognized and rebuilt the next time make.pl is run. Previously, you had to delete the first data.db database file.

v5.03.29 (March 29, 2005)

  1. New Feature: "merch:" URL prefix added. Can add "merch:" in front of non-affiliate URL's to convert them into affiliate URL's. For exampale, with DySE::InstrumentPro, can use HREF="merch:http://www.instrumentpro.com/C-Contact_us" in your templates and it is automatically converted to HREF="http://www.instrumentpro.com/C-Contact_us?source=Affiliates&kbid=AFFILIATEID" with your affiliate ID.

    Note: If a merchant does not support linking to arbitrary URL's, the the merch: prefix is ignored and the specified URL is used as is without any affiliate ID. So if you use the merch: prefix, you should verify that the URL displayed by view.pl contains your ID. If it does not, then instead you might want to link to the merchant's home page by using HREF="{merch.home}"

  2. New Feature: Can run "nicely" to reduce server load. If you use make.pl?nice=NUMBER, such as make.pl?nice=3, (NUMBER can be from 1 to 60, inclusive), the make.pl script will run for 1 second then pause for NUMBER seconds, then continue again. Thus if you use nice=3, then the script will run for 1 second, sleep for 3 seconds, run for 1 second, sleep for 3 seconds, etc. until completion, thus effectively having a duty cycle of 1/4 or 25%.

  3. Improvement: Checks to see if affiliate ID has changed. If the affiliate ID has changed (e.g.: from no key to having a key), make.pl will recognize the change and will remake the data.db database file. Previously, would have to manually delete the data.db database file.

  4. Improvement: Reports if /dyse directory exists. Reports if the /dyse directory exists (located in the root directory of your website). If it does not exist, then tries to create it and reports if it was created successfully or failed. The /dyse directory is used to display any graphics that the templates have (e.g.: "Buy" and "Click here for more info").

v5.03.12 (March 12, 2005)

E.&O.E.; © Cusimano.Com Corporation; www.c3scripts.com