First of all, I think these kinds of systems make tasks, whether combat-related or not, too easy. With an average ability score of "10" (on a traditional 3-18 range), this means that PCs are going to be successful 50% of the time on average. You could argue that the current edition of D&D employs mechanics which support an overall 70% success rate, but those feel, to me, to be inflated as well. There is too little chance of failure for my liking. Sure, there are going to be characters with low scores that only give a 30% chance of success (a score of "6"), but there are also going to be plenty of characters who have a 75% chance (a score of "15") at the same thing.
My other main issue with this kind of task resolution is that it stays static throughout the life of that character, regardless of the level of difficulty or PC experience. For those systems which simultaneously use an ability score advancement as characters level up, this can help PCs with low initial scores, but it also opens up the possibility for blessed characters to become unstoppable. Even though I've never been a fan of challenge ratings or difficulty class, the idea that attacking a dragon has the same chance for success as a kobold based on a Strength score is ludicrous.
If I'm not mistaken, this all traces back to (at least) B/X D&D's roll under ability score mechanic for any skill check or task not explicitly covered in the rules. Strangely enough, B/X uses OD&D's 1 on a d6, plus Strength mod to open stuck doors. With mods swinging from -3 to +3 this can give extremely strong characters a likely chance (4 in 6, or 66.6%) of success, yet still giving a 33% chance of failure. Chris Gonnerman (BFRPG author) uses a dice chain escalation of this Strength test mechanic as the challenge increases. Forcing a locked door is a 1 in d10 (plus Strength mod); bending iron bars a 1 in d20 (again, plus mod). This gives the GM more recourse when the player asks to attempt something nearly impossible.
Basic Fantasy also includes an Ability Roll mechanic which I've tested in play. Although initially skeptical, it works pretty well. PCs start with a standard DC target of 17 (on a d20 roll) to accomplish anything non-combat related, applying any relevant ability or situational modifiers. This target number goes down as the characters level up, representing a character's experience at work. I've heard people complain that a DC of 17 is too hard for level 1 characters and that they won't feel that they are even baseline competent. What's interesting is that a DC of 17 in 20 is a 20% chance of success whereas OD&D's 1 in 6 chance is a only 16.67% chance.
The key difference between these two ability test methods is that although the old d6 test has a tougher base chance of success, the ability mods themselves are more significant with each bonus equaling another 16.67%. By contrast, BF's Ability Roll mechanic only gives an extra 5% for each mod. This is the reason Gonnerman still used the d6 mechanic for Open Door/Feats of Strength tests - so that those bonuses would give more differentiation. One downside to the d20 roll is that everything is scaled in 5% increments. For those times you want more nuance, a dice chain of d6, d8, d10, d12, d20, and d100, etc. gives you that flexibility.
Conclusion: I understand why this is such a popular success/failure mechanic. I really do. It's simple to grasp and employ. However, in actual practice, I think it breaks verisimilitude beyond a point I can accept. The d6 (and its dice chain variants) are where I think I'm going to go for any skill or ability check in the future.
