Connect with us

Shared bus

Discussion in 'Electronic Design' started by Paul Camilleri, Feb 2, 2004.

Scroll to continue with content
  1. Hello group,

    Does anyone know if it is possible to have two or more PC's (running Windows
    2000) share an external IDE or SCSI device and if so any available designs
    or ideas? I know it can be done and solutions are available commercially but
    at a high cost. The reason I ask is that I need to experiment with NT
    clustering but my budget will mostly be blown when the OS and licences are
    purchased.

    Regards

    Paul Camilleri
     
  2. Ken Taylor

    Ken Taylor Guest

    Multiple masters on the bus will make for fun - not impossible, but why not
    use the network?

    Ken
     
  3. SCSI will do it out of the box, but I wouldn't advise it. There
    is no way to keep the processors from stomping all over each
    other on all disks connected to that SCSI channel. BTW, a system
    with an external SCSI connection is not a secure system (for the
    above reason). Of course any system that isn't physically secure
    is at risk too, but...
     
  4. I thought about mapping the relevant drives through the network for the
    quorum etc., but my guess is that it will not work - but I going to give it
    a try anyway. Hell if I am wrong and it does work then I'm quids in right?

    Paul
     
  5. I take it you are not trying to used clustered DB's?

    I have seen clustering done over tcpip but it was done using specialied
    software. Definately possible. How about using a raid that is connected to
    two machines?
     
  6. I take your point - thanks. I've set up (for clients) external SCSI shared
    storage for Windows NT. The security was no different from that which is
    normally available between an OS and a drive partition. But that was a SCSI
    backplane with Compaq proliant servers hanging off it and sharing terabytes
    of storage. Typical setup started about £750,000. Not in my league to
    finance for myself by a long shot. My budget is about £100 - no joke. But I
    do need to develope a cluster 'aware' application for Windows NT 2000
    Advanced Server.

    I'm starting to wonder if USB or Firewire is worth investiagting for this.

    Paul
     
  7. Ken Taylor

    Ken Taylor Guest

    The nice thing too about the network is that you are less constrained for
    where you physically place devices on the bus. However I know durn well I
    couldn't get this to work, so I can only wish you well from here!

    Cheers.

    Ken
     
  8. This at least confirms my own doubts. Thanks. My gut feel at this time is to
    look at Firewire (this bus is shareable as I understand it) with a simple
    IDE shared drive+interface (to experiment) and writing a device driver -
    wonderfull!

    I've got a lot to do on this as you guys can see but I will come back here
    with my findings if anyone is interested.

    Regards

    Paul Camilleri
     
  9. Hardware-wise Firewire will be your best bet, however I really wonder
    why you are looking so hard at available hardware technology. Your main
    problem will be on the software side; you don't seem to realize that.
    Harddrives do not have any arbitration facilities built in. And they
    can't, because they would need to understand filesystems. A filesystem
    needs to control allocation of clusters, and maintains fairly
    sophisticated data structures on disk for this. Who is going to be
    responsible for the validity and consistency of those when two computers
    are talking to the drive at the same time? How do you prevent one
    computer from reserving a cluster that has just been reserved by the
    other computer?

    Network file systems are built from the ground up to deal with this
    problem. Harddisks aren't. Don't even think about the hardware before
    you have solved this problem.
     
  10. Thanks for the reply. It is because I do realize what is involved that I am
    seeking an off-the-shelf product. Regarding the filesystem (at one level)
    and cluster/sector access (on another level) I agree these are obviously
    critical issues - and that would be the domain of the kernel level device
    driver. Here is my overall view at the moment:

    I/O Device My Device* IDE
    Request <--> Driver <--Firewire--> Controller <--> Device
    ------------------- ----------------------
    Computer 1-n Host

    *This could be a circuit or another computer running another device driver.

    Although the filesystem issues you mention are allways valid, the above
    'should' work in such a way as to completely circumvent them.

    I may be wrong (this is my 30000 foot view after all) but I doubt that
    quorum files are accessed at the lower cluster/sector levels but I will need
    to confirm this. If I am wrong then a rethink on my part is in order.

    Indicators:
    If you look at some of these USB memory sticks usaully available for
    notebook computers, some of them present themselves as a storage device with
    attendant drive letter etc., and accessed via an installable device driver.
    To me this is a clue as to the way forward.

    Please feel free to tear this idea down because I am only going to get one
    shot to come up with something that warrants further development.

    Regards

    Paul Camilleri
     
  11. Yes I do but at this stage I do not indtend to support any low level IO such
    as that used in earlier versions SQL server. However, if the idea is sound
    and initial results are promising, then disk sector access must be
    supported.

    Regards

    Paul Camilleri
     
  12. Sorry I did not respond fully to your question. Shared Raid. Probably the
    best solution but it falls way outside my budget.

    Regards

    Paul Camilleri
     
  13. I gladly accept the invitation ;-)

    Firstly, do not take USB memory sticks as an indication for anything.
    They're never used by more than one computer at a time. Sharing works by
    unplugging them from one computer and plugging them into another. Before
    unplugging them, you typically have to close the device (or unmount it,
    depending on the vocabulary used by your favourite operating system).
    This is not what you are heading for, so don't let yourself get misled.

    Secondly, your 30000 foot view is not detailed enough for assessing the
    situation. From a 5000 foot distance, you start to recognize that
    there's a whole hierarchy of drivers involved when an I/O-request gets
    processed. One of those is a filesystem driver. The device driver sits a
    couple of levels below this, and has no idea about the filesystem used.
    On the device driver level, you are reading and writing sectors or
    clusters. For this reason a device driver is not in a position to
    properly arbitrate between simultaneous requests from two different
    computers to the same harddrive. You can make sure they don't transfer
    data at the same time, but that's not even 5% of a solution.

    You need to make sure that filesystem data structures remain consistent,
    you need to ensure that the two systems are not caching different data,
    you need to ensure that the two systems are not trying to open the same
    file exclusively. How are you going to do this on a software level that
    doesn't understand filesystems? I'd say this is impossible to get right.

    The problem can only be solved satisfactorily on the filesystem level,
    and the solution is a network filesystem. You are in a position to judge
    how much work you're facing once you have understood what a filesystem,
    and especially a network filesystem (also called distributed filesystem)
    does, and why. You've got some reading to do first.
     
  14. Thanks for the information Stefan. I am already fully aware of the points
    you mentioned along with a truck-load of issues you did not mention. They
    are all valid but they have also all been successfully implemented/resolved
    by every shared-bus vendors's product to date ;)

    Regards

    Paul Camilleri
     
  15. Have they? Have you got an example?
     
  16. There are many examples to be found in the 'Microsoft Windows 2000 Data
    Centre Hardware Compatability List' on their site. I personally know of a
    large number I have implemented myself at client sites but NDA's prohibit
    giving further details here.

    Now at this point my guess is we are talking at crossed purposes or I have
    either missed a point or not correctly conveyed one. What do you think? I'm
    still hoping my idea (given the work required which is not trivial in the
    least) can still stand scrutiny at this stage. If not then I'll have to
    accept it and rethink.

    Regards

    Paul Camilleri
     
  17. So you want /me/ to search for examples of what /you/ think are products
    that solve your problem? You claim to have worked with them yourself, so
    why don't you just pick a good example you are familiar with? If it is a
    product listed in the HCL you won't breach any NDA by naming it.
    Indeed I have difficulties with your point. You stated in your first
    message of the thread that you wanted to share IDE or SCSI drives
    between several PC's. It goes without saying that you're trying to allow
    simultaneous access to the drive, because sharing by plugging the drive
    into different computers at different times is trivial and supported by
    any external harddisk. A shared bus wouldn't be necessary in this case.

    You claim that there are commercial solutions available for it, but you
    find them too expensive. You have so far failed to produce an example,
    but the only solutions I am aware of are special storage servers
    connected via network. They come in various sizes, but I would agree
    with you that they're fairly expensive. They all work on the network
    filesystem level, and they contain a computer including some operating
    system, hence the price.

    This does not appear to be what you want, since you want to work at the
    driver level. I assert that this is not going to work, and I don't know
    of any implementation that attempts to do it in this way. If any of your
    examples does, please name one.

    You were trying to find ideas and designs, yet claim to know commercial
    products that implement it. Have you studied any of them? Is there
    anything in their design you think you can do better?
     
  18. In my original question I did not ask for any help with solving 'my
    problem' -just advice which I duly got. Please do not presume to be able to
    do so.

    I did not provide the reference to the HCL in the hope you would do my work
    for me. That is not how I do things. In any event, that type of approach
    would more likely than not yield typically redundant, knee-jerk induced
    information if this thread is anything to go by.

    Here is one of many entries from the said HCL:

    Compaq ProLiant Cluster HA F100/F200 (DL380)
    Server 1: Compaq ProLiant DL380
    Server 2: Compaq ProLiant DL380
    SCSI/Raid Controller - Compaq StorageWorks Fibre Channel Host Adapter/P and
    Compaq StorageWorks 64-bit/66-Mhz Fibre Channel Host Adapter
    Shared SCSI/RAID Storage - Compaq StorageWorks RAID Array 4000/4100

    Note: "Shared SCSI/RAID Storage" - this means "Shared SCSI/RAID Storage".

    To 'fail' is to 'attempt' but not 'succeed'. I did not 'attempt' to provide
    an example, so therefore I did not fail. Please see above HCL entry for an
    example of the difference.

    Another 'claim' of mine is that NDA means "None Disclosure Agreement" - put
    simply - it means just that. Quoting a (public) HCL entry does not break
    that in the least. Given the simple terms I have used thus far, I fail to
    see how confusion has arisen and much less the assumption that an original
    respondant does not know what the term means either!

    Within this same thread I recieved other responses to my simple question and
    they got it right first time - no need to rephrase it in my opinion.

    With reference to my original posting (again) you will see that the above
    HCL entry example is obviously suitable from a technical point of view, but
    the cost is prohibitive - for me. I am just after doing it much more
    cheaply. NFS is irrelevant to my question and redundant in light of it - I
    suggest you read up on your own references, but in addition, read about the
    role of a DLM (Distributed Lock Manager) within a Windows 2000 cluster.

    Again, my original question did not make any mention of locking requirements
    for the simple reason that the I don't have any at this stage, but if I did
    I would realize a fundamental fact about the 'Windows Clustering Service'
    (more reading here for you I'm afraid) - it only supports the 'share
    nothing' model, and hence the need for the DLM which in itself ties in
    nicely with the driver development - which I've been doing for more than 20
    years. Was that another claim?

    Please try a little less grand-standing and pay more attention to what
    people are actually 'saying' before attempting an interpretation all by
    yourself.

    In light of the quality of your response I feel there is no point in
    pursuing this point any further.


    Regards

    Paul Camilleri
     
  19. Sorry, but I can't just swallow that. I don't like to be told off with
    subsequent communication shutdown. Ignore it if you like. We probably
    won't end up agreeing on anything, so the judges will have to be the
    silent witnesses.

    It appears to me that you are guilty of your own accusation - not paying
    attention to what people are saying. Here's why:

    o I did not presume to be able to help you solve your problem. Rather
    the opposite: I was trying to prevent you from going down a dead end. In
    my understanding, this qualifies as advice - precisely what you asked for.

    o I know perfectly well what NDA means. I don't see any confusion that
    has arisen from that. We both agree that it is no obstacle for quoting
    from the HCL.

    o You made a rather broad-line statement that the issues that I brought
    up had been solved by a number of manufacturers. As you didn't back that
    up with any evidence, I asked for an example. You didn't provide one,
    pointing to the HCL instead. That got me somewhat annoyed, I have to
    admit. If my usage of the word "failed" constitutes a semantic error, I
    have to apologize for that. I'm not a native English speaker, but since
    I specifically asked for an example, I believed the usage of the word
    "failed" was justified when you didn't provide one. If I believed wrong
    I gladly accept the correction. Now you finally provided a concrete
    example, which leads directly to the next point:

    o The Compaq RAID Array you mentioned can indeed be connected to two
    cluster nodes simultaneously. But the documentation of the Windows 2000
    cluster services make it rather clear that the two nodes can not control
    a single shared disk array at the same time (see
    http://www.sql-server-performance.com/wndows2000_clustering_install_step1.asp).
    This is in line with what I said.

    o I realize now that I may have misunderstood you. Probably you do not
    strive to allow this simultaneous control by two nodes. I thought you
    did, hence all this talk about filesystem data structure consistency and
    such. In this sense we might have talked cross-purposes, but I have to
    say that the way you reacted to my criticisms did nothing to make this
    clear - rather the opposite. It was specifically your mentioning of the
    USB stick as a clue to the way forward which made me believe that you
    didn't know what you're talking about. An I still can't fit this in with
    your 20 years driver writing experience. But maybe that's just my ignorance.

    I didn't want to offend you at all, believe me. But I don't think i'm
    100% to blame that this has blown up.
     
Ask a Question
Want to reply to this thread or ask your own question?
You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.
Electronics Point Logo
Continue to site
Quote of the day

-