Application-Specific

[ARC0112358]

Any word on the 2010 DWG format playing nicely with the Steelheads? I know Riverbed and Autodesk were working on this more closely for the upcoming release... Can anyone expand on this?

[gizbri]

Autodesk came out with a Hotfix for 2009 products

http://usa.autodesk.com/adsk/servlet/ps/dl...;linkID=9240618

[blancolam]

Matt,

The question here is: "Did Silver Peak really come up with some bizarre magic that transferred infinitely more data instantly"? Or is it a case whereby they compared apples-to-oranges and there really is no "bizarre magic" and it's just all smokescreen? It is only fair that you compare like-for-like.

Also, ask yourself this: why did SP choose a 10Mbps link instead of your typical T1 or 2Mbps? Refer to my comments about time reduction != data reduction and you'll understand.

In the end, our objective is very simple: be open and honest. We've shown our numbers and we're asking our competitors to do the same. When we have like-data, you can then draw your own conclusion.

Blanco

QUOTE(Matt Dwyer @ May 15 2008, 07:26 AM) 1022

Blanco,

As a user, as long as tests are real world, why do I care about ANYTHING except my time (ok, and my money!)? If they came up with some bizarre magic that transferred infinitely more data instantly, would that be bad? I would switch to their product in a heartbeat in that case (well, see money comment)... Seriously, though, I don't understand this high-pitched drilling on data reduction.

Matt Dwyer

[mattdwyerva]

Blanco,

As a user, as long as tests are real world, why do I care about ANYTHING except my time (ok, and my money!)? If they came up with some bizarre magic that transferred infinitely more data instantly, would that be bad? I would switch to their product in a heartbeat in that case (well, see money comment)... Seriously, though, I don't understand this high-pitched drilling on data reduction.

Matt Dwyer

QUOTE(Blanco Lam @ May 5 2008, 12:35 PM) 998

Jeff,


So, for the fourth time (a very broken record by now ), show us your data reduction numbers from your InfoWorld test. You have been more than willing to share your time reduction statistics and yet you've been very consistent in avoiding the publication of the data reduction statistics. Why?


Blanco

[blancolam]

Jeff,

In the latest Tech Target article (http://searchstorage.techtarget.com/news/article/0,289142,sid5_gci1312254,00.html#), you said:

Jeff Aaron, Silver Peak director of marketing, claimed in an email to SearchStorage.com that the numbers in Riverbed's report don't jibe with numbers Silver Peak has seen in the field, and questioned the Taneja Group's objectivity. "Make no mistake this is a Riverbed report with jacked-up numbers," Aaron wrote. "This is by no means a valid third-party verification."

Let's leave the Taneja Group out of the picture for now. You claimed that the numbers published in the report do not jibe with what SP has seen in the field and that Riverbed jacked up the numbers. It's understandable that you feel that way as the tests were done by Riverbed and I would feel the same if I were in your shoes. However, if you feel so strongly about this, then post your numbers so that everybody can do a fair comparison. I'm sure you realize that your argument is rather weak when you accuse Riverbed of jacking up the numbers but you have nothing (i.e. data reduction numbers) to substantiate it.

Now let's talk about the "Taneja Group's objectivity". You/Silver Peak asked InfoWorld to validate your result in this article: http://weblog.infoworld.com/tcdaily/archiv..._change_to.html. and then you questioned Taneja Group's objectivity in validating Riverbed's result? How do you justify InfoWorld being more objective than Taneja Group in this situation?

I also want to take the opportunity to address how Riverbed "jacked up" the numbers. If you recall from the article "Cisco WAAS customer uncovers AutoCAD file save issue" (http://searchstorage.techtarget.com/news/article/0,289142,sid5_gci1310518,00.html), the customer was quoted saying:

On the final tests, the save with an ISP of 50 left 904 KB unoptimized, meaning the system achieved a 90.05% optimization. But the save with an ISP of zero left about 10 MB unoptimized, a 9.43% optimization. "I can't really explain what that says," Gill said. "But in our day-to-day operations we use an ISP of 50, so we haven't seen a difference."

Now, cross check against the results matrix under 2007 DWG/10Mbps/Major Change/Cisco/ISP=0 and you'll see that the Riverbed tested result shows 10% for data reduction. Cisco WAAS customer tested with 10.2MB AutoCAD file and Riverbed tested with an 8.1MB AutoCAD file, and yet both arrived at virtually the same result. I think the result speaks for itself.

So, for the fourth time (a very broken record by now ), show us your data reduction numbers from your InfoWorld test. You have been more than willing to share your time reduction statistics and yet you've been very consistent in avoiding the publication of the data reduction statistics. Why?

If you have nothing, then may I suggest we lay this issue to rest as I feel we've wasted enough time on this and let's work on a solution with Autodesk instead. If you do have something, bring it forward, so that we can all take a look at it.

One final remark. In your email to Beth Pariseau, (http://storage.blogs.techtarget.com/2008/05/05/silver-peak-objects-to-tests-by-riverbed/), you said:

The fact that we show negative data reduction in some examples and that Riverbed comes in first in EVERY single test should be the first indicator that these results are biased and incorrect

That's an interesting logic. So, by your logic, the fact that a Ferrari beats a Toyota every time is because the results are biased and incorrect and it's not remotely possible that perhaps Ferrari is a better car and therefore consistently outperforms a Toyota? By the way, if you look under 2007 DWG/Major Change/ISP=0/10Mbps, you will see that SP's save time is the same as Riverbed. Biased?

It is time to be honest with the community and stop playing with cheap PR.

Kind regards,

Blanco

[blancolam]

http://blocksandfiles.com/article/5013

If you're interested in downloading the whitepaper, go to www.riverbed.com/autocadwp

[blancolam]

If you're wondering whether Cisco WAAS is also affected by this issue, here's the answer:

http://searchstorage.techtarget.com/news/a...1310518,00.html

Not too long ago, Cisco claimed that WAAS was immune to this AutoCAD issue (http://searchstorage.techtarget.com/news/article/0,289142,sid5_gci1303852,00.html). However, contrary to what they claim, they are equally affected by this AutoCAD issue.

[blancolam]

Jeff,

>> In essence, what you're saying is that, at no point was Silver Peak delivering < 20% time reduction and at times, Silver Peak was able to deliver up to 90%
>>time reduction for both 1Mbps and 10Mbps links on a 14MB DWG file. Is that an accurate statement??
>
> Yes, that is what I am saying.

Great. In that case, could you post the result for the 1Mbps testing?


> The purpose of those tests were to demonstrate application response time across the WAN, which is the problem being expressed by AutoCAD end users.
> Data reduction numbers were not the focus, although we typically observe that those numbers are on par with the time reduction percentages. We don't do
> caching or any application-layer tweaks, so it is safe to say that these results are primarily due to our deduplication technology.

If the results are, like you say, primarily due to your deduplication technology, then the data reduction number is very relevant. Afterall, isn't deduplication the reduction of data? The data reduction number becomes even more important when you have low bandwidth links like 1Mbps.

And I quote from the Infoworld article:

"Riverbed is affected by the AutoCAD file format/ISP setting issue more so than rival Silver Peak, and Silver Peak would like the world to know. To be clear, Riverbed will still accelerate AutoCAD DWG files over the WAN, even on a full save, but the data reduction is less than what Silver Peak is able to accomplish. The AutoCAD file problem is a very specific use case and not an indicator of any larger technology issue with Riverbed. Silver Peak's data deduplication technology is able to handle this particular problem much more gracefully than Riverbedâ⠬⠢s."

So the question boils down to this: what is the data reduction number for Silver Peak for the 14MB file with ISP=0?

At the risk of sounding like a broken record, can you please post the numbers for the baseline and when you ran the test with ISP=0? This will enable everybody to understand the effectiveness of Silver Peak's deduplication technology and the effect it has with AutoCAD 2007/08. Once again, I want to re-iterate the fact that time reduction is not the same as data reduction. I'm not saying that time reduction and data reduction are not related because they certainly can be given the right circumstances. However, what I am saying is that it makes no sense to take the number for time reduction and then assume that same number for data reduction.

Case in point: When downloading files from a domain controller with SMB Signing turned on (enabled and required), Riverbed Steelheads, cannot perform latency optimization but it can perform data reduction. This means that it's entirely possible to have 90% data reduction rate and yet the time reduction may be close to 0%. Reason for the lack of time reduction? CIFS protocol chattiness. However, this chattiness has nothing to do with the Steelhead's ability to deduplicate the data.

The reverse is also true. Given a big pipe, it's entirely possible to see a 90% time reduction, and yet the data reduction rate may be close to 0%. In this case, deduplication is virtually ineffective but other optimizations kick in to reduce the time it takes to complete the operation(e.g. removing CIFS chattiness). Note that it's possible to have a negative data reduction rate (i.e. sending more data than without using the WDS appliance) and yet still experience time reduction.

As you can see, if you take the time reduction number and assume it's the same for data reduction (or vice-versa), you'd come to the wrong conclusion.

Also, I know that Silver Peak can optimize CIFS traffic as well. Wouldn't the CIFS optimization help with removing the chattiness of CIFS and make the save operation of AutoCAD quicker?

Best regards,

Blanco

[aaronje]

Blanco -

< In essence, what you're saying is that, at no point was Silver Peak delivering < 20% time reduction and at times, Silver Peak was able to deliver up to 90% time reduction for both 1Mbps and 10Mbps links on a 14MB DWG file. Is that an accurate statement??>

Yes, that is what I am saying.

The purpose of those tests were to demonstrate application response time across the WAN, which is the problem being expressed by AutoCAD end users. Data reduction numbers were not the focus, although we typically observe that those numbers are on par with the time reduction percentages. We don t do caching or any application-layer tweaks, so it is safe to say that these results are primarily due to our deduplication technology.




Josh -

Good catch! Infoworld transcribed the chart wrong, which we missed on our end. We ve asked them to fix it.

Below is the accurate table, which shows the real cold pass numbers. It should be updated shortly online

Incremental Save Percentage 0% 50% 100%
Base line 40 65 62
Cold pass 39 62 61
Optimized save (warm pass) 23 20 19
Optimized save (hot pass) 23 20 18
Improvement 43% 69% 70%





Jeff

[joshtseng]

Jeff,

The results you reported on Infoworld are not only incomplete (no data reduction statistics), but what you did report strongly supports our view that Silver Peak is also affected by the Autocad 2007/2008 formatting changes to the same extent as Riverbed, if not more so. The problem was never about cold pass performance--Riverbed delivers 1st-transfer performance that is at least as good, if not better than those you reported through Infoworld. Rather, the problem Riverbed's customers encountered with Autocad 2007/2008 formatting changes was that "hot pass" performance was not significantly faster than cold pass performance.


Rather, the results you reported in Infoworld clearly showed that Silver Peak's hot pass performance is nearly identical to cold pass performance for Autocad 2007/2008. This indicates that Silver Peak's Network Memory technology is not recognizing the same data during the hot pass because those bytes have been scrambled by Autocad 2007/2008. Furthermore, it is interesting to note that raising the Incremental Save Percentage (ISP) to 100% had a very minor effect by improving Silver Peak's performance by one second. That is significantly less than what would be expected when using Riverbed.

Josh Tseng
Riverbed Technology