Most Popular Posts

Dec 20, 2014

Introducing OneSql Client ADO.NET 0.1 (alpha)

I’ve made two releases this week. The important one is OneSql.Client.AdoNet 0.1 (alpha). That is an ADO.NET adapter on top of OneSql.Client. There are a few important things to keep in mind when you try to adopt it:

Dependencies

The OneSql.Client.AdoNet package needs these two other packages:

Supported API

Unless you stick exclusively to -Async methods, you are likely to get a NotSupportedExecption. There are two messages – one for each of the reasons:

  • Synchronous operations are not supported.
    This message means that this call may require a roundtrip to the server which cannot be done synchronously. If there is an –Async version of this method, use that. Otherwise, try to find a workaround that doesn’t utilize this method.
  • This feature is not supported.
    This message means that OneSql Client hasn’t implemented the necessary part of the TDS protocol to enable this feature. You may search this blog to find out what is supported and what is not. For this kind of exceptions, you should be able to find a workaround.
    If you feel strongly about a missing feature, feel free to send me an email at zlatko+onesql@michailov.org. That doesn’t mean I’ll agree to implement it right away, but if I hear from a good number of people, I may do so.
Entity Framework

Enabling Entity Framework is the ultimate test for an ADO.NET provider. I will really appreciate your effort to migrate an EF-based app to Windows Store/Phone using OneSql Client ADO.NET. I’m sure you’ll get NotSupportedException’s please send me those call stacks. I’ll truly appreciate that.

 

The second release is an update on OneSql Client itself. It contains 2 bug fixes plus retargeting both Windows Store and Windows Phone. I expect more such updates as testing on the ADO.NET adapter continues.

I’ll continue posting notifications about new releases on https://twitter.com/OneSql. Follow it if you want to stay informed.

Oct 17, 2014

OneSql Client 1.0

OneSql Client is no longer a beta!

There’ve been 50 downloads of the beta version. I wish there were 100, but 50 is also a good number.

I didn’t receive any feedback, so I assumed the product was rock solid and no key features were missing.

OneSql Client 1.0 is a product now. You’ll find it at the same NuGet location - http://www.nuget.org/packages/OneSql.Client/.

The license also remains the same – BSD-2-Clause.

Questions and comments are still welcome at: zlatko+onesql@michailov.org

Apr 27, 2014

OneSql Client 0.3 (beta) Is Available

OneSql Client 0.3 (beta) is now available!

Same locations:

New in 0.3

The most significant addition to the OnceSql Client project is the test suite. The entire source code of the test project has been released. I may be used as samples.

There are no new features in this release. Various bugs were discovered and fixed during the course of adding test automation.

There are two breaking changes:

  • The timeout options have been removed. This is an API change that may require changing existing app code. Otherwise it may not compile. And of course, tasks that used to time out, no longer do so.
  • Service DONE sections are no longer exposed to apps. This is a behavior change. It will not prevent existing app code from compilation, but you may have to remove any SqlClient.MoveToNextResultAsync() calls you have added to skip over those “unwanted” results.
Next

The “beta” label will be removed when all of the following conditions are met:

  • The beta release has been available for at least 1 month, i.e. no earlier than May 27, 2014.
  • There are at least 100 downloads of a single beta package on NuGet.
  • All customer-reported issues for which there is no viable workaround have been fixed.
Support

All support content will continue to be published at this blog. This link http://bit.ly/1khQm90 queries OneSql Client content.

The recommended source, however, is https://twitter.com/@OneSql where along with references to blog posts, you’ll also find short status updates and news.

Feedback

To report an issue, to request a feature, or to provide feedback, please send an email to zlatko+onesql@michailov.org

Apr 18, 2014

OneSql Client – Breaking Change: Timeout Options Removed

This is an advance notice of an upcoming breaking change.

Summary

The following properties of type OneSql.Client.Options have been removed:

  • ConnectTimeout
  • CommandTimeout
  • DefaultConnectTimeout
  • DefaultCommandTimeout

OneSql Client operations will complete only based on [MS-TDS] protocol flows, not based on any timeout.

Version

0.3 and above.

Details

The purpose of timeouts is to give control back to the app when an operation takes too long to complete. This is mainly useful in synchronous API where the client thread is blocked on the operation.

In asynchronous API, like OneSql Client, the calling thread is never blocked. It is a developer’s choice whether to await the completion of the operation or to perform some other action.

The purpose of the timeout options was to provide a “timed await” mechanism that is common for all languages.However, the current implementation is incorrect – it leaves the SqlClient (and its underlying TdsClient) in a state that prevents further usage.

The cost of cleaning up SqlClient’s state exceeds the value of the feature by far. Moreover, that cleanup (which involves a new request to the server as well as receiving its response) may take a time that is long enough to defeat the purpose of the timeout. Therefore, timeouts have been discontinued.

Action Needed

Remove code that is setting (or getting) the above properties explicitly. Otherwise, when you upgrade your copy of OneSql Client, your app won’t compile.

If you want to get control after a certain time before the operation has finished, use standard Windows RT mechanisms specific to the language of your app:

C#

var cts = new CancellationTokenSource(secs * 1000);
await sqlClient.XxxAsync(…).AsTask(cts);

JavaScript

var sqlPromise = sqlClient.XxxAsync();
WinJS.Promise.timeout(secs * 1000, sqlPromise).then(…, …);

When you get control before the operation has finished, the SqlClient will be in a dirty state. You won’t be able to continue using it. Dispose of it. Any uncommitted transactions will be rolled back. Create a new SqlClient, connect it to the SQL Server, and decide how to continue.

Mar 29, 2014

OneSql Client is Back On Track!

Long story short - the problem I was hitting with C# was neither in Windows RT, nor in OneSql.Client. So I’m back on my way to implement the necessary test automation.

Meanwhile, I discovered a product bug – OneSql Client misses the end of the result set when multiple rows are read at a time. While I’ve fixed this bug already, I’ll hold off a bits update. The workaround for this bug is to read rows one by one. If this is a problem for any of the current adopters, please let me know via email at zlatko+onesql@michailov.org.

Mar 21, 2014

OneSql.Client in C# Apps

Update (Mar 29): This issue is now resolved. See: OneSql Client is Back On Track!.

I’ve been testing OneSql.Client using a JavaScript app assuming that if it works for JavaScript, it should work for C# as well since OneSql.Client is written in C#. I’ve been wrong.

I recently discovered that OneSql.Client doesn’t function properly when it’s hosted by C# apps. I’ve narrowed the problem down to the platform’s StreamSocket. I posted a question on the C# Apps forum, but nobody from the product team has responded yet. I have little hope anybody will ever respond.

This is very unfortunate, because C# represents a large portion of the Windows Store apps. Also, I still plan to implement the necessary test automation using C# in order to gain sufficient confidence in the quality of the product.

I first apologize to all of the adopters who can’t use my product in their C# apps. I should have tried C# before announcing the alpha releases.

I’m asking you for clues how I should be initializing/using StreamSocket to make it behave like it does in JavaScript apps. I admit I’m not certain whether the platform classes are exactly the same for JavaScript apps and for C# apps. I also suspect there might be differences in the threading models that I haven’t seen documented. Any other clues are also welcome.

Please email your suggestions to: zlatko+onesql@michailov.org.

Mar 9, 2014

OneSql Client 0.2 (alpha)

OneSql Client 0.2 (alpha) is now available. Same locations as before:
New in 0.2
This release fixes the error reporting model. I had noticed the WinJS app was not getting errors when I released 0.1. I thought I was doing something wrong that could be easily fixed. It turns out the WinRT interop layer can only surface a single HRESULT value from a component to a WInJS app, i.e. the exception instance – its type, message, stack, etc. is lost.
To fix that WinRT deficiency, I had to replace throwing exceptions with raising custom errors. That is now done through a special Error object that is exposed on both SqlClient and TdsClient. An app has to examine that Error property upon a promise error. The Error object contains a Code and a Message properties.
Next
Notice that this release is still “alpha”. It will remain like that until I have a sufficient coverage through test automation. That is the next thing on my list.
I will appreciate feedback. Please send email to zlatko+onesql@michailov.org.

Feb 26, 2014

Systems of Linear Equations 1.4 – Free Edition with Full Functionality

Systems of Linear Equations v1.4 is now available. The big news is that Free Edition has full functionality! Yes, you get complete, detailed, solutions for free.
Now that the only advantage of the paid edition is that you don’t get to see ads, I’ve reduced the price of Standard Edition to the minimum possible – $1.49.
Notice that these new releases are only available for Windows 8.1.
Start by installing the Free Edition.
If you want to get rid of the ads, try the Standard Edition.

Feb 11, 2014

Devices and Services

The term “devices and services” has become very popular recently. However, a lot of people including software engineers wonder what it means in terms of technology as well as how it will generate revenues for software companies.

This article represents my personal opinion and only my personal opinion. It may or may not match Microsoft’s opinion. Either way, this article MUST NOT be taken as official or sanctioned by Microsoft.

Devices

Price Wars

It is naïve to think that any company [except Apple] can generate considerable revenues from hardware devices. The reason for that is that devices are made of mass market components. If one company can design a device with certain capabilities and a certain form factor, then other companies can do the same at the same or lower cost. Thus a device pioneer won’t enjoy glory for too long. (The only exception could be Apple that has uniquely loyal customers who are willing to pay extra for the brand.)

Look at this offer – Nokia Lumia 521 without contract for $69! This is a great device that runs the exact same Windows and apps that run on any expensive Windows phone, and the best of all features - no blood sucking contract! I don’t think Nokia/Microsoft will make a ton of money out of this device at this price.

Or, how about this Dell Venue 8 Pro for $279? This device is a full blown computer that runs the real Windows, not RT.Will Dell make billions from it? I doubt it.

Open Platform

The PC platform was designed to be open from the very beginning. No one has ever owned it – neither chip makers, nor operating system vendors, nor PC manufacturers. Apps have had full access to the hardware capabilities of the PC. Users have been able to install apps from anywhere. Very democratic.

Democracy, however, is not easy to consume. It requires knowledge and responsibility. For instance, when a user needs some functionality, she has the freedom to choose any app. Great! But where to find apps? How can she be certain the app will not do something bad either intentionally or unintentionally?

The open platform works fine for computer/software savvy people. As long as you know where to search for apps, and you accept the risk of eventually installing a bad app, you will enjoy the open platform. Although there were hundreds of millions of PCs sold worldwide, it was a tiny fraction of the human population that was using them effectively.

Closed Platform

It is possible to create a platform that restricts what apps and users can do, but that platform can only enforce such restrictions over the layers built on top of itself. If the lower layers of the stack are open, an alternative platform could be installed that would not respect those restrictions, and thus no guarantees could be made. Therefore, in order for a platform to preserve integrity, it must include all the lower layers of the stack down to the very bottom – the hardware device.

That’s where devices come to play a major role. Devices themselves won’t be generating revenues. They will be part of a closed platform that is expected to be more profitable than its open predecessor.

Why will the closed platform be more successful than the open platform? Because, although we are crazy about freedom, we are too lazy to sustain it. We want simple, constrained, experiences. We want someone else to guarantee the safety of our own device. And that’s exactly what the closed platform offers – when you need certain functionality, there is a single app store where you can possibly find an app. All apps have been tested by the platform vendor before they’ve been made available for you to download. When an app runs on your device, the platform won’t let it use any device capability that it hasn’t declared. When you want to remove an app, the platform will wipe it out like it has never been there. That perfectly fits the needs of our modern spoiled civilization.

But how would a close platform generate a revenue? Here is the model that Apple pioneered in modern days: the platform vendor takes a percentage from every single app sale. Let’s say 4 developers have made app sales for $1M each. If the platform vendor takes a 25% cut from each app sale, it will make $1M while each of the developers will make $750K. Now imagine 4 million developers selling apps. You can do the math. Not every developer will be successful? No problem. The platform vendor requires an upfront license fee from every developer whether he will become successful or not. And yes, that license has to be renewed every year. Clever, isn’t it?

Devices and Services

The key factor that contributes to the success of a closed platform is the user experience. A good user experience attracts users who are potential app buyers. A large number of potential app buyers attracts app developers who make more apps available on the platform. More apps make the platform even more attractive to users which closes the loop and so does the process continue.

Emerging User Experience

Of course, each platform vendor will put its signature on the user experience, and will provide a different set of free apps to make its platform unique, but that’s not the interesting part. The important thing is the emerging user experience that is the same across all device platforms, and that will change the software industry.

To understand this new experience, let’s take a look at the evolution of doing business: In-person –> The Web –> Devices and Services.

In-Person
  1. Get your butt off the couch to sit in your car.
  2. Drive to the point of business.
  3. Stand in line.
  4. Do your business.
  5. Drive back.
  6. Place your butt where it belongs – on the couch.

5 out of those 6 steps are plain overhead. Not only does that waste time, but it’s often dangerous.

The Web

The Web significantly simplified the experience by replacing physical driving with cyber navigation:

  1. Get your butt off the couch to sit by the PC or to fetch the laptop.
  2. Sit in the car and drive to the place of business.
    Navigate to the target web site.
  3. Stand in line.
  4. Do your business.
  5. Drive back.
  6. Place your butt where it belongs – on the couch.

The Web made it safer and faster. But one big problem still remains – we are lazy, and we want to sit on the couch doing nothing.

That’s where devices and services comes to help.

Devices and Services
  1. Get your butt off the couch to sit by the PC or to fetch the laptop.
    Suspend the Facebook app.
  2. Sit in the car and drive to the place of business.
    Navigate to the target web site.
    Bring up the app for the job.
  3. Stand in line.
  4. Do your business.
  5. Drive back.
  6. Place your butt where it belongs – on the couch.
    Resume the Facebook app.

Your butt remains planted on the couch the whole time. Halleluiah! This experience can only be successful.

Impact on the Web Site Industry

I’m not writing out of excitement about how our society will get fat faster. I’m writing because I foresee a disruptive change coming up triggered by devices and services.

From *SP to Plain HTML

The vast majority of web sites is implemented using a server page technology - ASP.NET, JSP, PHP, etc. An HTTP request comes to the server and gets dispatched to the web site’s handler. The handler loads and executes a custom module that does some computation and ultimately sends back HTML that a browser visualizes in front of the end user.

Since the user experience is what sells a web site, the UI dictates how code is written and structured. You may see projects where all the code is packaged in server page modules, though most commonly developers try to extract business logic into separate modules. That’s the funny part. Since all code paths are driven by UI experiences, it’s hard to draw a clear line where UI-specific code ends and where business logic starts. Thus what typically ends up in the so called business logic can be classified in two buckets:

  • Unrelated utility functions.
  • An object model that represents the persistence schema.

This is funny because despite of the developers’ best intents, such a separation has no value with regard to reusability or scalability. It might as well live with the rest of the server page code because that’s the only purpose it serves.

Now the important part - that code doesn’t belong to the server side at all! Its place is on the client side just like any other app. That’s the first architectural change that devices and services introduces:

Devices and services kicks web UI out of the server, and converts it to plain HTML. The web site becomes yet another client app for the given service.

This is a very important change because it deeply affects the COGS of running a web site. Instead of the service vendor paying for the resources (CPU, memory, bandwidth, etc.) needed to generate a web page, each user will run that code in their own browser. And those utility libraries and object models? They’ll have to be rewritten in JavaScript - as I already explained, they only serve the UI.

Scale Out and the Current Lack of It

The removal of the UI from the server will reveal an unspoken truth:

Most web sites never really scaled out.

(There is a tiny portion of modern web sites, mainly in the search or social space, that are designed to scale out and that I don’t include in the above statement.)

Most development teams actually believe their web sites scales out. And their scale out is done by replicating front end boxes where those teams believe the most compute-intense code, the business logic, runs. First, we now know that all that front end code doesn’t belong on the server side to begin with. Second, the assumption that the most compute-intense code is outside of the data layer is wrong, because the relevance of a web site is proportional to the data it processes. If there was no giant volume of data around which the whole experience gravitates, you’d be running that functionality on your local box!

(In all fairness, there are some web sites that offer a poor man’s backend scale out by creating static partitions. A user session statically belongs to exactly one partition based either on the host name or on the user account. While this approach does offer some scalability, the scalability is uneven and the approach requires constant monitoring and a manual migration of data among partitions to maintain a relatively even distribution. Thus the cost of this approach is too high to accept as long-term solution.)

That defines a problem for the whole web site industry:

Follow the search and social leaders and design your data stores to scale out.

A lot has been written about Big Data, and I won’t repeat it. My point is that Big Data stores will become the primary choice of data stores for web-facing systems. The sooner development teams accept this idea, the better position they will get in the new world of devices and services.

Thin Client No More

Lastly, I’d like to point out that our perception of the client experiences will change. So far we’ve been categorizing clients as “thin” (web pages) and “rich” (standalone apps.) The web interface was viewed as lass capable than standalone apps. That perception is about to change – device apps will provide limited UI for limited functionality suitable for the lower resolution and the smaller form factor of the device while the web pages will provide full functionality leveraging high definition screens. For instance, we’ll be using our phones to read our Facebook feeds most of the time, and we’ll use a browser when we want to do a more sophisticated operation like managing settings.

Conclusion

Devices and services launches a new era in software history. The trend is irreversible. We, software developers, will have to accept the constraints and the requirements of the new platform.

I am truly excited about the disruption to web-facing systems. As a backend developer, I’ve been waiting for the demand to build up, and now that the moment has come, and I can’t hide my excitement.

Jan 14, 2014

OneSql Client – World’s First SQL Server Provider for Windows Store Apps

If you work on a SQL Server client apps, you may have noticed that it’s not possible to connect directly to SQL Server from a Windows Store app. Instead, you have to develop a web service, you have to deploy it somewhere in the cloud, you (or your customers) have to pay for running it, and lastly, you have to support it. And all that is because there is no SQL Server provider for Windows Store apps.
Today, I am excited to announce OneSql Client – world’s first SQL Server provider for Windows Store apps!
Overview
OneSql Client is a Windows Runtime component that can be used from any Windows Store app regardless of its language. I wrote a demo app in JavaScript to prove that concept (see link below.)
OneSql Client is freely available as a NuGet package at http://www.nuget.org/packages/OneSql.Client/ or in a raw form from its home location http://onesql.michailov.org/.
Please be advised that this is an early alpha release and it may not behave correctly in many cases. At this point, it is only intended for experimental and learning purposes.
OneSql Client implements the [MS-TDS] protocol from scratch. Well, it doesn’t implement the whole protocol yet. The purpose of this alpha release is to prioritize the remaining work.
API
OneSql exposes API from two namespaces:
  • OneSql.Client.Sql – this namespace contains the actual SQL Server provider, SqlClient, that is ready for consumption by apps. It returns rows as JSON arrays.
  • OneSql.Client.Tds – this namespace contains low-level primitives that could be used to implement your own provider. However, if you need a feature that SqlClient doesn’t offer, I strongly recommend that you first request that feature from SqlClient before trying to implement your own provider.
Limitations
  • Supports SQL Server 2012 or higher. It might be possible to work against SQL Server 2008 and 2008 R2, but that hasn’t been tested.
  • Doesn’t support the following data types:
    • image, text, ntext – these types have been superseded by varbinary(max), varchar(max), and nvarchar(max) respectively.
    • decimal/numeric – these types are too big for JavaScript.
    • UDT – I don’t think there is a base support for these in Windows Runtime.
    • sql_variant – I’ve never needed this type. Hopefully not too many people will be crying for it.
    The above types can still be used in the storage schema or in server-side code. They just can’t be retrieved directly. You’ll have to CONVERT/CAST the respective column to a supported type.
  • Only SQL batches are supported, i.e. parameterized queries are not yet supported. They are on the plan, just not yet.
Getting Started
Download and open the SampleWindowsStoreApp project. Play with it. You’ll most certainly find bugs. Please describe your repro clearly, zip any prerequisite SQL and send it to zlatko+onesql@michailov.org.
You may also see how connections are established, how rows are being fetched, and how to move to the next result set. The documentation is very Spartan at the moment. It is on my plan to provide samples.
Known Issues
Errors encountered by OneSql get lost somewhere among the awaits and the JavaScript promises. I know it is annoying. I’m working on that too.
Support
The best place to get the latest news, updates, and references to resources is https://twitter.com/@OneSql
Articles will continue being published at http://zlatko.michailov.org/.
New releases and other executable artifacts will be published at https://onesql.michailov.org/.
For everything else, send me an email at zlatko+onesql@michailov.org.