Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[BrianTRice@gmail.com]

I've written some data processing code to try to take Zero's parts listings and add metadata to them and emit some listings on the unofficial manual.


The output pages, currently, are listed here and are highly subject to deletion and re-organizing. Do not link to them expecting them to be stable yet.


https://zeromanual.com/wiki/Category:GeneratedPartsPage


I'm looking for comments about the structure and usefulness of these pages.


I know for example that I'm not listing prices, and I'm not sure that I can reasonably do that, so don't expect that I'll just add them because you want them. The unofficial manual isn't a dealer and doesn't have an official license for this data, so I'm publishing this under Fair Use / Derivative Work claims and will likely add a real disclaimer to this section of the website to avoid trouble.


I also know that the way the articles are carved up is a little troublesome. I can recombine the pages into larger ones, and can even merge the tables, but I'm unsure how useful that would be.


I also know that there are typo issues and some mangled text in there, as a result of the complicated method by which I extract the text from the parts catalog documents. There's a professional process of data cleaning to improve on here, which just will take me time to get right. It took a while to make the output this clean.


I have a few thoughts about how this might go in general, which might be the right food for thought for you to provide feedback on:
- Each part number might get its own page (huge number of pages, but leaves rooms for people to comment and makes it easy to link to/between part numbers).
- The "component" articles could each include the relevant parts listing table.
- Indexing articles by part type (bolts, brackets, etc).
- Splitting the articles back into per-model-year articles (mirrors the original structure but that makes it harder to track changes across years and models which is what I'm trying to do here).


The nice part is that the program that generates these pages can wipe these clean and start over to generate something else. But that's why I'm asking aloud for feedback.

[Crissa]

Like, the mirrors on the mid-year 2014 match newer mirrors, but that has a range of part numbers.  How are you going to handle that?

-Crissa

[BrianTRice@gmail.com]

Like, the mirrors on the mid-year 2014 match newer mirrors, but that has a range of part numbers.  How are you going to handle that?

This is the result of an automated build process I apply to the official documents, to extract their texts and organize it.

The output for the mirrors is on this page (likely moving to a Gen2 subpage soon):
2014: https://zeromanual.com/wiki/Bodywork_Parts
2015+: https://zeromanual.com/wiki/Body_Trim_Parts

The language for this is "supersedes" and lists part numbers. I will try to cross-hyperlink those entries. I will also make a reasonable effort to try to get related parts on the same page. In the case of mirrors, there's a nomenclature issue in the parts diagrams that I have to normalize to get good results. A lot of what I'm doing is using heuristics to identify items.

[Keith]

Having the year ranges makes it much easier to know whether some salvaged parts might fit your machine, this is extremely useful, don't split them.

It would also be great to have information about equivalent interchangeable parts from other manufacturers, but that's going to be sparse and hard to find. For example I know this about the rear sprocket for my 2016 FX, don't know how I found it: equivalent to '03 - '07 KTM 85 SX (132mm diameter M8 4-bolt circle, 110mm center hole dia.)

[BrianTRice@gmail.com]

Having the year ranges makes it much easier to know whether some salvaged parts might fit your machine, this is extremely useful, don't split them.

It would also be great to have information about equivalent interchangeable parts from other manufacturers, but that's going to be sparse and hard to find. For example I know this about the rear sprocket for my 2016 FX, don't know how I found it: equivalent to '03 - '07 KTM 85 SX (132mm diameter M8 4-bolt circle, 110mm center hole dia.)

Getting the year ranges was one of my major goals with this, so that makes sense.

This is the result of an automated process, so I cannot interleave arbitrary content easily. Remember, I still have component pages which do have replacement information (even if it's not consistently associated with the page). What I do intend and need to do is link between the part number and the component pages.

Basically, I need to use hyperlinks heavily to solve problems that ordinarily require more manual content labor, of which I'm in short supply.

[Crissa]

Could we make the automation into sections of pages instead of specific pages?

That would make it easier to find the year ranges if the 'bot just added a section for every component it comes across from each year that rough a previous years' search term.  Try to reduce the overhead somehow by playing a little match game.

-Crissa

[BrianTRice@gmail.com]

Could we make the automation into sections of pages instead of specific pages?

That would make it easier to find the year ranges if the 'bot just added a section for every component it comes across from each year that rough a previous years' search term.  Try to reduce the overhead somehow by playing a little match game.

The pages are coded to be transcluded into other pages (which I use extensively to maintain the manual as small pages now), but yes the library I'm using has the ability to edit a section of a page.

I cannot parse what you're suggesting, though. I run the script across all of the parts documents and it always rebuilds the entire page set in about 15 minutes (and most of that is the bot pausing to avoid overloading the site with edits).

If you literally want to look up parts by year, we already have that, though.

[Crissa]

I think you parsed it right.

If you ran each script to build on the prior year and only split out the diff... Then manually find what changed names.  Even better if you could build a list of 'keywords' for each part to teach the script to find matches better the next year.

That way if I'm looking at 'mirror' or 'signal relay' I don't have to worry about ending up on the wrong page (unlike doing it by parts number).

-Crissa

[BrianTRice@gmail.com]

I think you parsed it right.

If you ran each script to build on the prior year and only split out the diff... Then manually find what changed names.  Even better if you could build a list of 'keywords' for each part to teach the script to find matches better the next year.

That way if I'm looking at 'mirror' or 'signal relay' I don't have to worry about ending up on the wrong page (unlike doing it by parts number).

Oh, you want (1) to compare over years, and (2) topic/keyword retrieval. I have something for the change comparison, but I'm not sure how to present the results yet. I can bump that up the to-do list, sure, since I'd also like to solve that.

I do have a keyword arrangement (detects the "Type" value which labels things bolts or brackets or whatever), but applying the keywords requires a set of heuristics that takes time to refine.

The manual in general struggles immensely with trying to help people find items. I really want people to use the search field at the top right of every page, so I make redirect pages with all the terms/synonyms people use, but I have never been able to measure what people want that it doesn't deliver.

[Crissa]

Yeah!

Ahh.  Do you have a log of search queries?  It can't be very large over time, can it?

-Crissa

[BrianTRice@gmail.com]

I do not own the site, and the site owner was a real pain about SSH or log access, only letting me access the site through SFTP where the logs were not kept. He cut off SFTP access when I pointed out a security flaw in the hosting, so I can't even upgrade mediawiki from 1.27 which is quite old or install new modules now.

I have a somewhat drastic backup plan I've implemented but haven't thrown the switch, as it would require everyone knowing about it, which is a big PR stink.

So, no, I have no idea how much traffic the site gets or how people use the site.

[BrianTRice@gmail.com]

FYI I fixed a design issue in the text processing algorithms which was cutting off and mangling some text. The tables are now quite a bit more complete, and I removed all the "N/A" entries which are artifacts of how the drawings are produced and not helpful in the data.

[Crissa]

Brian, if you need to set up a Patreon or something to defray hosting costs, just ask.

You're giving us a massive value here.  I would not have even considered buying a Zero without it.

And it wouldn't be the only website I toss coins into the kitty for.

-Crissa

[BrianTRice@gmail.com]

I got a very nice set of donations for the aftermath of last year's accident that I guess I can technically write into the new DSR, at least. But yes, sooner or later, there will be a fundraising conduit that if we're all lucky will support a nonprofit some day when electric motorcycles are a large enough market.

At present, I'm just hoping a side business around this stuff develops to justify it and keep the loop going around engineering circles.

[BrianTRice@gmail.com]

I think you parsed it right.

If you ran each script to build on the prior year and only split out the diff... Then manually find what changed names.  Even better if you could build a list of 'keywords' for each part to teach the script to find matches better the next year.

That way if I'm looking at 'mirror' or 'signal relay' I don't have to worry about ending up on the wrong page (unlike doing it by parts number).

Oh, you want (1) to compare over years, and (2) topic/keyword retrieval. I have something for the change comparison, but I'm not sure how to present the results yet. I can bump that up the to-do list, sure, since I'd also like to solve that.

I do have a keyword arrangement (detects the "Type" value which labels things bolts or brackets or whatever), but applying the keywords requires a set of heuristics that takes time to refine.

The manual in general struggles immensely with trying to help people find items. I really want people to use the search field at the top right of every page, so I make redirect pages with all the terms/synonyms people use, but I have never been able to measure what people want that it doesn't deliver.

Well, I made a quick year-by-year history of the Zero SR model (2014-2019):
https://www.zeromanual.com/wiki/SR_Model/Parts_History

It's not very readable, a large number of links are broken, and it reveals some weaknesses in the parsing right now. BUT I would like someone to look at this and think about what might be better. I can break it down by diagram (physical areas of the bike) but that's all I can think of presently, which means I'll have to sleep on it.

Now I'll go try to deliver another project someone's waiting on...