The way that I saw people are using it around here is using the plugin as the main benchmarking tool to determine whether a theme is broken/not-broken without even considering the context of the errors generated by the plugin.
You have something of a point. The Theme Check plugin is very ‘dumb’ in certain ways. For example, it can’t actually read HTML or PHP code, it’s just doing various regular expression matches. If you happen to write a theme and leave out, say, the comment_form function, then it will find that out. But if you mention “comment_form” in a PHP comment, then it will be happily tricked by that.
Fundamentally, Theme Check assumes good intentions on the part of the theme author. If you want to fool it, then it’s childishly simple to trick it into thinking all is well. It will not discover intentionally hidden malicious code, because code is really infinitely malleable. It’s a lot like the halting problem in that way, you can’t discover if a program halts in a 100% decidable manner. And without human eyeballs, no program can determine the intent of another program.
Fortunately, none of this is the purpose of Theme Check. It’s meant to be a time-saver for our human reviewers, and a quick means for theme authors to not forget to do certain things. For example, it’s amusing how many themes were submitted that didn’t actually have a way to do post pagination. Like, you could never see older posts because they were missing the link to them. Theme Check looks for that, and says “hey, you forgot to include the next page function” and things of that nature. That’s like 80% of Theme Check, actually.
Theme Check also checks for common malicious code, such as base64_decode and eval and such. These functions are almost universally used for evil in the specific case of themes, and while they do have real-world uses (I have plugins that use each of those in real cases), they almost never have real-world uses in themes specifically. Themes are, by design, intended to do a very specific job. If you have code that does something outside of “displaying the site”, then it should probably be in a plugin, not a theme. Themes can have settings pages, and most themes have far too many settings IMO , but there’s never a real case where a theme should be evaluating PHP code, or base64 encoding/decoding data. For every case you can think of, there’s either a better way, or you should move it into a plugin.
So yes, Theme Check is dumb, but intentionally so. It’s a time saver. It helps theme authors make sure they didn’t forget something silly. It helps theme reviewers do a quick first-pass of a theme and find problems with it. It is used in the WordPress.org theme submission process, to make sure that themes don’t have any obvious security problems, and to prevent theme submissions from becoming overwhelming (the theme submitter form on WordPress.org will automatically disallow you from even uploading a theme if it fails the Theme Check).
But you’re right, there’s no substitute for the human eyeball when it comes to evaluating code and intentions. All of the themes on WordPress.org are reviewed by people. Theme Check is just a way for us to remove the low hanging fruit. If a theme author can’t even get their theme to pass Theme Check’s ridiculously dumb checks, then it’s probably not worth the valuable time of our reviewers to even look at.
rvision_ saidJSON encoding cannot handle a PHP class, but PHP serialization can handle it just fine. Although why in the world you would be copy and pasting actual classes from one site to another I have no idea. It makes a lot more sense to build an array of your settings instead for export/import cases.
I think the fear of base64 encoding comes from users with no OOP backgrounds. They still live in an ‘array’ world. Regarding JSON – can JSON serialize a php class? Or am I wrong here?
And nobody “fears” base64 encoding. It’s just used for evil much more often than it’s used for good. We banned it from WordPress.org because it is almost never used for good purposes and the few cases it was used in a sane manner can be easily rewritten to not need it.
BTW , Hi! I’m Otto. I wrote the first version of Theme Check. Although others wrote most of the actual checks, I wrote the base64 checking code, and unilaterally banned it from all themes on WordPress.org.
Why did I do that? Because with only one exception, base64 encoding was invariably used to hide malicious code, or link spam, or other malware that we did not want on the site. Use of base64_decode in a theme was a 99.99% reliable indicator of spam and/or malware. It is virtually NEVER used for a valid reason.
The only argument I’ve ever heard for the use of base64 encoding in a theme is the specific one given in this thread: export/import of settings. The argument goes something like this: I want the theme user to be able to copy/paste a big chunk of meaningless data from a form field into another form field on a different site. This will transfer the settings easily.
Now, that is an interesting use case, however I’ve never seen a realistic reason why base64 encoding must be used for this. I grant you that base64 ends up with a “form-safe” text to be used in a textarea or some such thing. However, if you’re not using classes, then using json encoding along with esc_textarea works perfectly well for transporting anything that isn’t a PHP class, and will cover almost all realistic cases. This also has the benefit of remaining readable to the user, so that they can see what they’re transporting across, instead of some large piece of gibberish.
You can use base64 encoding all you like in your theme, but here’s the simple truth: it is an almost universally reliable indicator of malware or spam, when it’s used in a theme. You may not be writing such, but it’s so commonplace in the real world that people will recognize it and call you out for it. Theme Check will continue to check for it, and it will continue to be banned from the free themes directory on WordPress.org. These are just simple facts. You don’t have to agree with them, but that won’t change them.